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Channel coding refers to the class of signal transformations designed to improve 
communications performance by enabling the transmitted signals to better with¬ 
stand the effects of various channel impairments, such as noise, interference, and 
fading. These signal-processing techniques can be thought of as vehicles for accom¬ 
plishing desirable system trade-offs (e.g., error-performance versus bandwidth, 
power versus bandwidth). Why do you suppose channel coding has become such a 
popular way to bring about these beneficial effects? The use of large-scale inte¬ 
grated circuits (LSI) and high-speed digital signal processing (DSP) techniques 
have made it possible to provide as much as 10 dB performance improvement 
through these methods, at much less cost than through the use of most other meth¬ 
ods such as higher power transmitters or larger antennas. 


6.1 WAVEFORM CODING 


Channel coding can be partitioned into two study areas, waveform (or signal 
design) coding and structured sequences (or structured redundancy), as shown in 
Figure 6.1. Waveform coding deals with transforming waveforms into “better 
waveforms,” to make the detection process less subject to errors. Structured se¬ 
quences deals with transforming data sequences into “better sequences,” having 
structured redundancy (redundant bits). The redundant bits can then be used for 
the detection and correction of errors. The encoding procedure provides the coded 
signal (w'hether waveforms or structured sequences) with better distance properties 
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Figure 6.1 Basic digital communication transformations. 






than those of their uncoded counterparts. First, we consider some waveform coding 
techniques. Then, starting with Section 6.3. we treat the subject of structured 
sequences. 


6.1.1 Antipodal and Orthogonal Signals 


Antipodal and orthogonal signals have been discussed earlier; we shall repeat the 
paramount features of these signal classes. The example shown in Figure 6.2 illus¬ 
trates the analytical representation, s^it) = = sin loor, 0<t< T, of a slnu.soidal 

antipodal signal set, as well as its waveform representation and its vector represen¬ 
tation. What are some synonyms or analogies that are used to describe antipodal 
signals^ We can say that such signals are mirror images, or that one signal is the 
negative of the other, or that the signals arc ISO*^ apart. 

The example shown in Figure 6.3 illustrates an orthogonal signal set made up 
of pulse waveforms, described by 








where p{t) is a pulse with duration t = TO, and T is the symbol duration. Another 
orthogonal waveform set frequently used in communication systems is sin x and 
cos X, In general, a set of equal-energy signals .9,(r), where / = 1, 2. ... , M, consti¬ 
tutes an orthonormal (orthogonal, normalized to unity) set if and only if 



for / = / 
otherwise 



Analytic 

representation 


Waveform 

representation 



si(^) = sin too^ 



Vector 

representation 


d = 2<E 



S 2 (t) = -sin (Oof 
0<t<T 




Figure 6.2 Example of an antipodal signal set. 
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Figure 6.3 Example of a binary orthogonal signal set. 


where Zjj is called the cross-correlation coefficient, and where E is the signal energy, 
expressed as 



The waveform representation in Figure 6.3 illustrates that 5i(f) and .92(/) cannot in¬ 
terfere with one another because they are disjoint in time. The vector representa¬ 
tion illustrates the perpendicular relationship between orthogonal signals. Consider 
some alternative descriptions of orthogonal signals or vectors. We can say that the 
inner or dot product of two different vectors in the orthogonal set must equal zero. 
In a two- or three-dimensional Cartesian coordinate space, we can describe the 
signal vectors, geometrically, as being mutually perpendicular to one another. We 
can say that one vector has zero projection on the other, or that one signal cannot 
interfere with the other, since they do not share the same signal space. 


6.1.2 yiFary Signaling 

With M-ary signaling, the processor accepts k data bits at a time. It then instructs 
the modulator to produce one of M = 2* waveforms; binary signaling is the special 
case where k = \. For > 1, M-ary signaling alone can be regarded as a waveform 
coding procedure. For orthogonal signaling (e.g., MFSK), as k increases there is 
improved error performance or a reduction in required E^IN^, at the expense of 
bandwidth; nonorthogonal signaling (e.g., MPSK) manifests improved bandwidth 
efficiency, at the expense of degraded error performance or an increase in required 
E^INq. By the appropriate choice of signal waveforms, one can trade off error 
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probability, £ft/No, and bandwidth efficiency. Such trade-offs are treated in greater 
detail in Chapter 9. 

6.1.3 Waveform Coding 

Waveform coding procedures transform a waveform set (representing a message 
set) into an improved waveform set. The improved waveform set can then be used 
to provide improved Pa compared to the original set. The most popular of such 
waveform codes are referred to as orthogonal and hiorrhogonal codes. The 
encoding procedure endeavors to make each of the waveforms in the coded 
signal set as unalike as possible; the goal is to render the cross-correlation coeffi¬ 
cient Zf ^—among all pairs of signals, as described by the integral term in Equation 
6.1—as small as pos.sible. The smallest possible value of the cross-correlation coef¬ 
ficient occurs when the signals are anticorrelated = -1); however, this can be 

4 

achieved only when the number of symbols in the set is two {M = 2) and the sym¬ 
bols are antipodal. Jn general, it is possible to make all the cross-correlation coeffi¬ 
cients equal to zero [1]. The set is then said to be orthogonal. Antipodal signal sets 
are optimum in the sense that each signal is most distant from the other signal in 
the set; this is seen in Figure 6.2 where the distance d between signal vectors is seen 
to be d = 2 VEt where E represents the signal energy during a symbol duration T. 
as expressed in Equation (6.2). Compared with antipodal signals, the distance 
properties of orthogonal signal sets can be thought of as "pretty good” (for a given 
level of waveform energy). In Figure 6.3 the distance between the orthogonal sig¬ 
nal vectors is seen to be d = V 2£- 

ITie cross-correlation between two signals is a measure of the distance be¬ 
tween the signal vectors. The smaller the cross-correlation, the more distant are the 
vectors from each other. This can be verified in Figure 6.2, where the antipodal sig¬ 
nals (whosie Zij ~ -1) are represented by vectors that are most distant from each 
other; and in Figure 6.3, where the orthogonal signals (whose z,, = 0) are repre¬ 
sented by vc(itors that are closer to one another than the antipodal vectors. It 
should be obvious that the distance between two identical waveforms (w'hose 
Zij = 1) is zero. 

The orthogonality condition in Equation (6.1) is presented in terms of 
waveforms s,{t) and Ay(t), where i, j = \ . M, and M is the size of the wave¬ 

form set. Each waveform in the set (.v,(r)) may consist of a sequence of pulses, 
where each pulse is designated with a level +1 or -1, which in turn represents the 
binary digit 1 or 0, respectively. When the set is expressed in this way. Equation 
(6.1) can be simplified by stating that (5,(01 constitutes an orthogonal set if and 
only if 


number of digit agreements - number of digit disagreements 

total number of digits in the .sequence 


f 1 for i = j 

lo otherwise 
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6.1.3.1 Orthogonal Codes 

A one-bit data set can be transformed, using orthogonal codewords of two 
digits each, described by the rows of matrix H, as follows: 


Data set 

0 

1 


Orthogonal codeword set 




For this, and the following examples, use Equation ( 6 . 3 ) to verify the orthogonality 
of the codeword set. To encode a 2 -bit data set, we extend the foregoing set both ^ 
horizontally and vertically, creating matrix H2. 


Data set 

0 0 
0 1 

1 0 
1 1 


Orthogonal codeword set 


H 


fo 0 

0 

0 

0 

1 

0 

1 

0 0 

1 

1 

0 

1 

1 

0_ 


H 

H 


I 


1 




The lower right quadrant is the complement of the prior codeword set. We 
continue the same construction rule to obtain an orthogonal set H3 for a 3 -bit 
data set. 


Data Set Orthogonal codeword set 
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( 6 . 4 c) 












H-) H") 

L* ^ ^ J 

1 

0 

0 

0 

0 

0 

0 

1 

1 

1 

1 



1 

0 

1 

0 

1 

0 

1 

1 

0 

1 

0 



1 

1 

0 

0 

0 

1 

1 

1 

1 

0 

0 



1 

1 

1 

- 0 

1 

1 

0 

1 

0 

0 

1 - 



In general, we can construct a codeword set H; 
Hadamard matrix, for a /c-bit data set from the 

,, of dimension 2* x 2*, 
_ j matrix, as follows: 

called a 



Each pair of words in each codeword set Hj, H2, H3,..., Hf ,,... has as many digit 
agreements as disagreements [ 2 ]. Hence, in accordance with Equation ( 6 . 3 ), z,y = 0 
(for / ^ y), and each of the sets is orthogonal. 
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Just as M-ary signaling with an orthogonal modulation format (such as 
MFSK) improves the performance, waveform coding with an orthogonally con¬ 
structed signal set, in combination with coherent detection, produces exactly the 
same improvement. For equally likely, equal-energy orthogonal signals, the proba¬ 
bility of codeword (symbol) error, can be upper bounded as [2] 

Pe{M) ^ (M - \) Q ( 6 . 5 ) 


where the codeword set M equals 2*, and k is the number of data bits per code¬ 
word. The function Q{x) is defined by Equation ( 3 . 43 ), and = kEf, is the energy 
per codeword. For a fixed M, as is increased, the bound becomes increasingly 
tight; for Pe( 1 ^) ^ 10 "^, Equation ( 6 . 5 ) is a good approximation. For expressing the 
bit-error probability, we next use the relationship between P^ and P^, given in 
Equation ( 4 . 112 ) and repeated here: 


Psik) 

PE{k) 



1 



PbW ^ M /2 

Pe{M) (M - 1 ) 



Combining Equations ( 6 . 5 ) and (6.6), the probability of bit error can be bounded 
as follows: 










6.1.3.2 Biorthogonal Codes 


A biorthogonal signal set of M total signals or codewords can be obtained 
from an orthogonal set of M /2 signals by augmenting it with the negative of each 
signal as follows: 



For example, a 3 -bit data set can be transformed into a biothogonal codeword set 


as follows: 


Data set 

0 0 0 
0 0 1 
0 1 0 
0 1 1 

1 0 0 
1 0 1 
1 1 0 
1 1 1 


Biorthogonal codeword set 



0 

1 

0 

1 


0 

0 

1 

1 



1 1 
0 1 
1 0 
0 0 
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The biorthogonal set is really two sets of orthogonal codes such that each code¬ 
word in one set has its antipodal codeword in the other set. The biorthogonal set 
consists of a combination of orthogonal and antipodal signals. With respect to z, , of 
Equation (6.1), biorthogonal codes can be characterized as 



( 6 . 8 ) 


One advantage of a biorthogonal code over an orthogonal one for the same 
data set is that the biorthogonal code requires one-half as many code bits per code¬ 
word (compare the rows of the B, matrix with those of the H3 matrix presented 
earlier). Thus the bandwidth requirements for biorthogonal codes are one-half the 
requirements for comparable orthogonal ones. Since antipodal signal vectors have 
better distance properties than orthogonal ones, it should come as no surprise that 
biorthogonal codes perform slightly better than orthogonal ones. For equally 

likely, equal-energy biorthogonal signals, the probability of codeword (symbol) 
error can be upper bounded, as follows [2]: 


which becomes increasingly tight for fixed M as EfJN^^ is increased. Pr{M) is a 
complicated function of we can approximate it with the relationship [2] 



The approximation is quite good for M > 8. Therefore, we can write 






( 6 . 10 ) 


These biorthogonal codes offer improved Pb performance, compared with the 

performance of the orthogonal codes, and require only half the bandwidth of 
orthogonal codes. 


6.1.3.3 Transorthogonal (Simplex) Codes 


A code generated from an orthogonal set by deleting the first digit of each code¬ 
word is called a transorthogonal or simplex code. Such a code is characterized bv 
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A simplex code represents the minitnum energy equivalent (in the error-probability 
sense) of the equally likely orthogonal set. In comparing the error performance of 
orthogonal, biorthogonal, and simplex codes, we can state that simplex coding re¬ 
quires the minimum F.h/N^^ for a specified symbol error rate. However, for a large 
value of M, all three schemes are essentially identical in error performance. 
Biorthogonal coding requires half the bandwidth of the others. But for each of 
these codes, bandwidth requirements (and system complexity) grow exponentially 
with the value of M: therefore, such codine schemes are attractive onlv when large 
bandwidths are available. 


6.1.4 Waveform-Coding System Example 


Figure 6.4 illustrates an example of assigning a A-bit message from a message set of 
size M - 2^, with a coded-pulse sequence from a code set of (he same size. Each 
A-bit message chooses one of the generators yielding a coded-pulse sequence or 
codeword. The sequences in the coded set that replace the messages form a wave¬ 
form set with good distance properties (e.g., orthogonal, biorthogonal). For the or¬ 
thogonal code deseribed in Section 6.1.3.1. each codeword consists of M = 2* pulses 
(representing code bits). Hence 2* code bits replace A message bits. The chosen se¬ 
quence then modulates a carrier wave using binary PSK, such that the phase (<t>^ = () 
or it) of the carrier during each code-bit duration, 0 < / < T,., corresponds to the am¬ 
plitude (/ = -l or 1) of the /th bipolar pulse in the codeword. At the receiver in Fig¬ 
ure 6.5, the signal is demodulated to baseband and fed to M correlators (or 
matched filters). For orthogonal codes, such as 


as those characterized by the 
Hadamard matrix in Section 6.1.3.1, correlation is performed over a codeword du¬ 
ration that can be expressed as 7'= 2^7",. For a real-time communication system, 
messages may not be delayed; hence, the codeword duration must be the same as 
the message duration, and thus, 7'can also be expressed as 7' = (log^A/) 7^ = kT^,. 


Transmitter 



Waveforms 


\ 





0 < ^ < r,. 


±K cos (Oof 

Phase modulation 
by dtj -0,n is 
equivalent to 
amplitude 
modulation by 
-1 and +1 


Figure 6.4 Waveform-encoded system (transmitter). 
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Reference signals 
Orthogonal pulse 
waveform set 




Generator 1 



cos 0)0^ 


Generator 2 




2 : 


Jo 




rid) = ±K cos (Dot -i- n(t) 

i = 2,.,., M 




Demodulation to 
baseband 


Generator M 




Receiver 



Where T^kTf, = 2*Tc 


mi 


Choose the 
waveform 
(or codeword) 
with the 
largest z, 


Figure 6.5 Waveform-encoded system with coherent detection (receiver). 


where is the message-bit duration. Note that the time duration of a message bit 
is M/k times longer than that of a code bit. In other words, the code hits or coded 
pulses (which are PSK modulated) must move at a rate M/k faster than the mes¬ 
sage bits. For such orthogonally coded waveforms and an AWGN channel, the ex¬ 
pected value at the output of each correlator, at time F, is zero, except for the 
correlator corresponding to the transmitted codeword. 

What is the advantage of such orthogonal waveform coding compared with 
simply sending one bit or one pulse at a time? One can compare the bit-error per¬ 
formance with and without such coding by comparing Equation (4.79) for coherent 
detection of antipodal signals with Equation (6.7) for the coherent detection of or¬ 
thogonal codewords. For a given size k-b\{ message (say, k = 5) and a desired bit¬ 
error probability (say, 10“^), the detection of orthogonal codewords (each having a 
5-bit meaning) can be accomplished with about 2.9 dB less EfJNo than the bit-by-bit 
detection of antipodal signals. (The demonstration is left as an exercise for the 
reader in Problem 6.28.) One might have guessed this result by comparing the per¬ 
formance curves for orthogonal signaling in Figure 4.28 with the binary (antipodal) 
curve in Figure 4.29. What price do we pay for this error-performance improve¬ 
ment? The cost is more transmission bandwidth. In this example, transmission of 
an uncoded message consists of sending 5 bits. With coding, how many coded 
pulses must be transmitted for each message sequence? With the waveform coding 
of this example, each 5-bit message sequence is represented by Af = 2* = 2^ = 32 
code bits or coded pulses. The 32 coded pulses in a codeword must be sent in the 
same time duration as the corresponding 5 bits from which they stem. Thus, the 


314 


Channel Coding: Part 1 Chap. 6 






required transmission bandwidth is 32/5 limes lhal of the uncoded case. In general, 
the bandwidth needed for such orthogonally coded signals is M/k times greater 
than that needed for the uncoded case. Later, more efficient ways to trade off the 
benefits of coding versus bandwidth [3, 4] will be examined. 


6.2 TYPES OF ERROR CONTROL 

Before we discuss the details of structured redundancy, let us describe the two 
basic ways such redundancy is used for controlling errors. The first, error detection 
and retransmission, utilizes parity bits (redundant bits added to the data) to detect 
that an error has been made. The receiving terminal does not attempt to correct 
the error; it simply requests that the transmitter retransmit the data. Notice that a 
two-way link is required for such dialogue between the transmitter and receiver. 
The second type of error control, /bnvnr^/ error correction (FEC), requires a one¬ 
way link only, since in this case the parity bits are designed for both the detection 
and correction of errors. We shall see that not all error patterns can be corrected; 
error-correcting codes are classified according to their error-correcting capabilities. 

6.2.1 Terminal Connectivity 

Communication terminals are often classified according to their connectivity with 
other terminals. The possible connections, shown in Figure 6.6, are termed simplex 
(not to be confused with the simplex or transorthogonal codes), half-duplex, and 
full-duplex. The simplex connection in Figure 6.6a is a one-way link. Transmissions 


Terminal 


Terminal 

A 


B 


Transmission in only one direction 

(a) 


Terminal 


Terminal 

A 

Or 

B 


Transmission in either direction, 
but not simultaneously 

(b) 


Figure 6.6 Terminal connectivity 
classifications, (a) Simplex, (b) Half¬ 
duplex. (c) Full-duplex. 
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are made from terminal A to terminal B only, never in the reverse direction. The 
half-duplex connection in Figure 6.6b is a link whereby transmissions may be made 
in either direction but not simultaneously. Finally, the full-duplex connection in 
Figure 6.6c is a two-way link, where transmissions may proceed in both directions 
simultaneously. 

6.2.2 Automatic Repeat Request 

When the error control consists of error detection only, the communication system 
generally needs to provide a means of alerting the transmitter that an error has 
been detected and that a retransmission is necessary. Such error control procedures 
are known as automatic repeat request or automatic retransmission query (ARQ) 
methods. Figure 6.7 illustrates three of the most popular ARQ procedures. In each 
of the diagrams, time is advancing from left to right. The first procedure, called 
stop-and-wait ARQ, is shown in Figure 6.7a. It requires a half-duplex connection 
only, since the transmitter waits for an acknowledgment (ACK) of each transmis- 
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Figure 6.7 Automatic repeat request (ARQ). (a) Stop-and-wait ARQ (half¬ 
duplex). (b) Continuous ARQ with pullback (full-duplex), (c) Continuous ARQ with 
selective repeat (full-duplex). 
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sion before it proceeds with the next transmission. In the figure, the third transmis¬ 
sion block is received in error; therefore, the receiver responds with a negative ac¬ 
knowledgment (NAK), and the transmitter retransmits this third message block 
before transmitting the next in the sequence. The second ARQ procedure, called 
continuous ARQ with pullback, is shown in Figure 6.7b. Mere a full-duplex connec¬ 
tion is necessary. Both terminals arc transmitting simultaneously; the transmitter is 
sending message data and the receiver is sending acknowledgment data. Notice 
that a sequence number has to be assigned to each block of data. Also, the ACKs 
and NAKs need to reference such numbers, or else there needs to be a priori 
knowledge of the propagation delays, so that the transmitter knows which mes¬ 
sages are associated with which acknowledgments. In the example of Figure 6.7b, 
there is a fixed separation of four blocks between the message being transmitted 
and the acknowledgment being simultaneously received. For example, when mes¬ 
sage 8 is being sent, a NAK corresponding to the corrupted message 4 is being re¬ 
ceived. In the ARQ procedure, the transmitter “pulls back” to the message in error 
and retransmits all message data, starting with the corrupted message. The final 
method, called continuous ARQ with selective repeat, is shown in Figure 6.7c. Here, 
as with the second ARQ procedure, a full-duplex connection is needed. In this pro¬ 
cedure, however, only the corrupted message is repeated; then, the transmitter 
continues the transmission sequence where it had left off instead of repeating any 
subsequent correctly received messages. 

The choice of which ARQ procedure to choose is a trade-off between the re¬ 
quirements for efficient utilization of the communications resource and the need to 
provide full-duplex connectivity. The half-duplex connectivity required in Figure 
6.7a is less costly than ful-duplex; the associated inefficiency can he measured by 
the blank time slots. The more efficient utilization illustrated in Figures 6.7b and c 
requires the more costly full-duplex connectivity. 

The major advantage of ARQ over forward error correction (FEC) is that 
error detection requires much simpler decoding equipment and much less redun¬ 
dancy than does error correction. Also, ARQ is adaptive in the sense that inlorma- 
tion is retransmitted onlv when errors occur. On the other hand, FEC mav be 
desirable in place of, or in addition to, error detection, for any of the following 
reasons: 


1. A reverse channel is not available or the delay with ARQ would be excessive. 

2. The retransmission strategy is not conveniently implemented. 

3. The expected number of errors, without corrections, would require excessive 
retransmissions. 


6.3 STRUCTURED SEQUENCES 


In Section 4.8 we considered digital signaling by means of M -2^ signal waveforms 
(A/-ary signaling), where each waveform contains k bits of information. We saw 


that in the case of orthogonal A/-ary signaling, we 


decrease Pn by increasing M 
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(expanding the bandwidth). Similarly, in Section 6.1 we showed that it is possible to 
decrease Pq by encoding k binary digits into one ol M orthogonal codewords. The 
major disadvantage with such orthogonal coding techniques is the associated ineffi¬ 
cient use of bandwidth. For an orthogonally coded set of M = 2*^ waveforms, the re¬ 
quired transmission bandwidth is M/k times that needed lor the uncoded case. In 
this and subsequent sections we abandon the need for antipodal or orthogonal 
properties and focus on a class of encoding procedures known as parity-check 
codes. Such channel coding procedures are classified as structured sequences be¬ 
cause they represent methods of inserting structured redundancy into the source 
data so that the presence of errors can be detected or the errors corrected. Struc¬ 
tured sequences are partitioned into three subcategories, as shown in Figure 6.1;# 
block, convolutional, and turbo. Block coding (primarily) is treated in this chapter, 
and the others are treated in Chapters 7 and 8 respectively. 

6.3.1 Channel Models 

6.3.1.1 Discrete IMemoryless Channel 

A discrete memoryless channel (DMC) is characterized by a discrete input 
alphabet, a discrete output alphabet, and a set of conditional probabilities P{j'i) 
(1 < / < M, \ <j< Q), where / represents a modulator A/-ary input symbol, j repre¬ 
sents a demodulator ^-ary output symbol, and P{ jH) is the probability of receiving 
J given that / w'as transmitted. Fach output symbol of the channel depends only on 
the corresponding input, so that for a given input sequence U = //,. /o,..,, u „,,.. ., 

Usj, the conditional probability of a corresponding output sequence Z = Zi^ . 

z,j, .z.v rtiay be expressed as 

/-(Zlu) = n />(zj»„,) (6.12) 

m ^ I 

In the event that the channel has memory (i.e., noise or fading that occurs in 
bursts), the conditional probability of the sequence Z would need to be expressed 
as the joint probability of all the elements of the sequence. Equation (6.12) ex¬ 
presses the memoryless condition of the channel. Since the channel noise in a mem¬ 
oryless channel is defined to affccl each symbol independently of all the other 

symbols, the conditional probability of Z is seen as the product of the independent 
element probabilities. 

6.3.1.2 Binary Symmetric Channel 

A binary symmetric channel is a special case of a DMC; the input and 

output alphabet sets consist of the binary elements (0 and 1). The conditional prob¬ 
abilities are symmetric: 

F(()|l) = F(1|()) =/7 

(6.13) 

F(l|l) = F(()|()) = 1 - p 
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Equation (6.13) expresses the channel transition probabilities. That is, given that a 
channel symbol was transmitted, the probability that it is received in error is p 
(related to the symbol energy), and the probability that it is received correctly is 
(1-/7). Since the demodulator output consi.sis of the discrete elements 0 and 1, the 
demodulator is said to make a firm or hard decision on each symbol. A commonly 
used code system consists of BPSK modulated coded data and hard decision de¬ 
modulation. Then the channel symbol error probability is found using the methods 
discussed in Section 4.7.1 and Equation (4.79) to be 



where is the channel symbol energy per noi.se density, and ^(.r) is defined in 
Equation (3.43). 

When such hard decisions are used in a binary coded system, the demodula¬ 
tor feeds the two-valued code symbols or channel bits to the decoder. Since the de¬ 
coder then operates on the hard decisions made by the demodulator, decoding with 
a BSC channel is called hard-decision decoding. 


6.3.1.3 Gaussian Channel 

We can generalize our definition of the DMC to channels with alphabets that 
are not discrete. An example is the Gaussian channel with a discrete input alphabet 
and a continuous output alphabet over the range (-», oo). The channel adds noise 
to the symbols. Since the noise is a Gau.ssian random variable with zero mean and 
variance cT, the resulting probability density function (pdf) of the received random 
variable z, conditioned on the symbol 7/;^ (the likelihood of Uj,), can be written as 



for all z, where k = 1,2,... ^ M. For this case, memoryless has the same meaning as 

it does in Section 6.3.1,1, and Equation (6.12) can be used to obtain the conditional 
probability for the sequence Z. 

When the demodulator output consists of a continuous alphabet or its quan¬ 
tized approximation (with greater than two quantization levels), the demodulator 
is said to make soft decisions. In the case of a coded system, the demodulator feeds 
such quantized code symbols to the decoder. Since the decoder then operates on 
the soft decisions made by the demodulator, decoding with a Gaussian channel is 
called soft-decision decoding. 

In the case of a hard-decision channel, we arc able to characterize the detec¬ 
tion process with a channel symbol error probability. However, in the case of a 
soft-decision channel, the detector makes the kind of decisions (soft decisions) that 
cannot be labeled as correct or incorrect. Thus, since there are no firm decisions, 
there cannot be a probability of making an error; the detector can only formulate a 
family of conditional probabilities or likelihoods of the different symbol types. 
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It is possible to design decoders using soft decisions, but block code soft- 
decision decoders are substantially more complex than hard-decision decoders; 
therefore, block codes arc usually implemented with hard-decision decoders. For 
convolutional codes, both hard- and soft-decision implementations are equally 
popular. In this chapter we consider that the channel is a binary symmetric channel 
(BSC), and hence the decoder employs hard decisions. In Chapter 7 we further dis¬ 
cuss channel models, as well as hard- versus soft-decision decoding for convolu¬ 
tional codes. 


6.3.2 Code Rate and Redundancy 

In the case of block codes, the source data are segmented into blocks of k data bits, 
also called information bits or message bits; each block can represent any one of 2^ 
distinct messages. The encoder transforms each A-bit data block into a larger block 
of n bits, called code bits or channel symbols. The {n - k) bits, which the encoder 
adds to each data block, are called redundant bits, parity bits, or check bits; they 
carry no new information. Fhc code is referred to as an {n, k) code. The ratio of re¬ 
dundant bits to data bits, denoted {n - k)fk, within a block is called the redundancy 

w 

of the code; the ratio of data bits to total bits, k/n, is called the code rate. The code 

rate can be thought of as the portion of a code bit that constitutes information. For 

example, in a rate \ code, each code bit carries \ bit of information. 

In this chapter and in Chapters 7 and 8 we consider those coding techniques 

that provide redundancy by increasing the required transmission bandwidth. 

For example, an error control technique that employs a rate 1/2 code (100% 

redundancy) will require double the bandwidth of an uncoded system. How'cvcr, if 

a rate 3/4 code is used, the redundancy is 33% and the bandwidth expansion is only 

4/3. In Chapter 9 we consider modulation/coding techniques for bandlimited 

channels where complexity instead of bandwidth is traded for error performance 
improvement. 


6.3.2.1 Code-Element Nomenclature 


Different authors describe an encoder’s output elements in a variety of wavs; 


code bits, channel bits, code symbols, channel symbols, parity bits, parity symbols. 
The terms are all very similar. In this text, for a binary code, the terms “code bit,” 
“channel bit,” “code symbol,” and “channel symbol” have exactly the same mean- 
ing. TTie terms “code bit” and “channel bit” are most descriptive for binary codes 
only. The more generic names “code symbol” and “channel symbol” are often pre¬ 
ferred because they can be used to describe binary or nonbinary codes equally well. 
Note that such code symbols or channel symbols arc not to be confused with the 


grouping of bits to form transmission symbols that was done in previous chapters, 
rhe terms “parity bit” and “parity symbol” arc used to identify only those code 
elements that represent the redundancy components added to the original data. 
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6.3.3 Parity-Check Codes 


6.3.3.1 Single-Parity-Check Code 


Parity-check codes use linear sums of the information bits, called parity symbols 
or parity hits, for error detection or correction. A single-parity-check code is con¬ 
structed by adding a single-parity bit to a block of data bits. The parity bit takes on the 
value of one or zero as needed to ensure that the summation of all the bits in the code¬ 
word yields an even (or odd) result. The summation operation is performed using 
modulo-2 arithmetic (exclusive-or logic), as described in Section 2.9.3. If the added 
parity is designed to yield an even result, the method is termed even parity; if it is de¬ 
signed to yield an odd result, the method is termed odd parity. Figure 6.8a illustrates 


a serial data transmission (the rightmost bit is the earliest bit). A single-parity bit is 


added (the leftmost bit in each block) to yield even parity. 

At the receiving terminal, the decoding procedure consists of testing that the 
modulo-2 sum of the codeword bits yields a zero result (even parity). If the result is 
found to be one instead of zero, the codeword is known to contain errors. The rate of 
the code can be expressed as kl{k -i-1). Do you suppose the decoder can automatically 
correct a digit that is received in error? No, it cannot. It can only detect the presence 
of an odd number of bit errors. (If an even number of bits are inverted, the parity test 
will appear correct, which represents the case of an undetected error. ) Assuming that 


Parity 

bit 



Figure 6.8 Parity checks for serial 
and parallel code structures, (a) Se¬ 
rial structure, (b) Parallel structure. 



parity check 


parity check 
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all bit errors are equally likely and occur independently, we can write the probability 
of } errors occurring in a block of n symbols as 



(6.15) 


where p is the probability that a dumnel symbol is received in error, and where 



(6.16) 


is the number of various ways in which j bits out of // may be in error. Thus, for a, 
single-parity error-detection code, the probability of an undetected error with a, 
block of n bits is computed as follows: 

/)/2(for/t even) 

{n - l)/2(lorM tiild) / \ 

j= 1 \d / 

iLxample 6.1 Even-Parity Code 

Configure a (4, 3) even-parity error-detection code such that the parity symbol 
appears as the leftmost symbol of the codeword. Which error patterns can the code 
detect? Compute the probability of an undetected message error, assuming that all 
symbol errors are independent events and that the probability of a channel symbol 
error is p - 10"\ 

Solution 


Message Parity Codeword 


000 

0 

0 

000 

1(X) 

1 

1 

100 

010 

1 

1 

010 

no 

0 

0 

no 

001 

1 

1 

001 

101 

0 

0 

101 

on 

0 

0 

on 

in 

1 

1 

111 



parity message 


The code is capable of delecting all single- and triple-error patterns. The probability 
of an undetected error is equal to the probability that two or four errors occur any¬ 
where in a codeword. 


^nd = (2)^^^' (4)^^ 

= 6p"(l - p)- -f 
= bp- - \2p^ + Ip* 

= bdtr-')^ * 12(10"^"V + 7(irj"-’)'' ^ 6 X 10"^ 
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6.3.3.2 Kectan};ular Code 

A rectangular code, also called a product code, can be thought of as a parallel 
code structure, depicted in Figure 6.8b, First we form a rectangle of message bits 
comprising M rows and N columns; then, a horizontal parity check is appended to 
each row and a vertical parity check is appended to each column, resulting in an 

augmented array of dimensions {M + 1) x (W + 1). The rate of the rectangular code 
k/n can then be written as 


k ^ MN 

n " (M+ l)(/V-h 1 ) 

How much more powerful is the rectangular code than the single-parity code, 
which is only capable of error detection? Notice that any single bit error will cause 
a parity check failure in one of the array columns and in one of the array rows. 
Therefore, the rectangular code can correct any single error pattern since such an 
error is uniquely located at the intersection of the error-detecting row and the 
error-detecting column. For the example shown in Figure 6,8b, the array dimen¬ 
sions are M - N = 5; therefore, the figure depicts a (36. 25) code that can correct a 
single error located anywhere in the 36-bll positions. For such an error-correcting 
block code, we compute the probability that the decoded block has an uncorrected 
error by accounting for all the w'ays in which a message error can be made. Starting 
with the probability of j errors in a block of n symbols, expressed in Equation 
(6.15), we can write the probability of a message error, also called a block error or 
word error, for a code that can correct all t and fewer error patterns as 

/’«= i; (-/>)"■'■ (6.]8) 

y-;>I \ J / 

where p is the probability that a channel symbol is received in error. For the exam¬ 
ple in Figure 6.8b, the code can correct all single error patterns (/ = 1) svithin the 

rectangular block of rt = 36 bits. Hence, the summation in Equation (6.18) starts 
with j - 2: 





When p is reasonably small, the first term in the summation is the dominant one; 
1 herefore, for this (36,25) rectangular code example, we can write 



The bit error probability Pp depends on the particular code and decoder. An ap¬ 
proximation for Pp Is given in Section 6.5.3. 


6.3.4 Why Use Error-Correction Coding? 

Error-correction coding can be regarded as a vehicle for effecting various system 
trade-offs. Figure 6.9 compares two curves depicting bit-error performance versus 
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E^/Ao- O'lt; curve represents a typical modulation scheme without coding. The 
second curve represents the same modulation with coding. Demonstrated below 
are four benefits or trade-offs that can be achieved with the use of channel coding. 

6.3.4.1 Trade-Off 1: Error Performance versus Bandwidth 

Imagine that a simple, inexpensive voice communication system has just been 
developed and delivered to a customer. The system does not use error-correction 
coding. Consider that the operating point of the system can be depicted by point A 
in Figure 6.9 = 8 dB, and P^ = 10'^). After a few trials, there are complaints 

about the voice quality; the customer suggests that the bit-error probability should 
be lowered to 10The usual way of obtaining better error performance in such a 
system would be by effecting an operating point movement from point A to, say, 
point B in Figure 6.9. How'ever, suppose that the E^IN(^ of 8 dB is the most that is 
available in this system. The figure suggests that one possible trade-off is to move 
the operating point from point A to point C. That is, “walking” down the vertical 
line to point C on the coded curve can provide the customer with improved error 
performance. What docs it cost? Aside from the new components (encoder and de¬ 
coder) needed, the price is more transmission bandwidth. Error-correction coding 
needs redundancy. If we assume that the system is a real-time communication 
system (such that the message may not be delayed), the addition of redundant bits 
dictates a faster rate of transmission, which of course means more bandwidth. 

6.3.4.2 Trade-OfT2: Power versus Bandwidth 

C'onsider that a system without coding, operating at point D in Figure 6.9 
- 14 dB, and Pp = 10"^), has been delivered to a customer. The customer has 
no complaints about the quality of the data, but the equipment is having some 
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reliability problems as a result of providing an of 14 dB. In other words, the 

equipment keeps breaking down. If the requirement on or power could be 

reduced, the reliability difficulties might also be reduced. Figure 6.9 suggests a 
trade-off by moving the operating point from point D to point £. That is. if error- 
correction coding is introduced, a reduction in the required can be achieved. 
Thus, the trade-off is one in which the same quality of data is achieved, but the 
coding allows for a reduction in power or £,,/%. What is the cost? The same as 
before—more bandwidth. 


Notice that for non-real-time communication systems, error-correction coding 
can be used with a somewhat different trade-off. It is possible to obtain improved 
bit-error probability or reduced power (similar to trade-off 1 or 2 above) by paying 
the price of delay instead of bandwidth. 


6.3.4.3 Coding Gain 


The trade-off example described in the previous section has allowed a reduc¬ 
tion in from 14 dB to 9 dB, while maintaining the same error performance. 

In the context of this example and Figure 6.9, we now define coding gain. For a 


given hit-error prohuhiiity\ coding gain is defined as the “relier or reduction in 


£/% that can be realized through the use of the code. Coding gain G is generally 
expressed in dB. such as 


,6.K, 

where {EJN^d„ and {Ef,IN^X represent the required uncoded and coded, 

respectively. 


6.3.4.4 Trade-Off 3: Data Rale versus Band\iidtli 

Consider that a system without coding, operating at point D in Figure 6.9 
(£/,/AZ(, = 14 dB, and ^’) has been developed. Assume that there is no prob¬ 

lem with the data quality and no particular need to reduce power. However, in this 

example, suppose that the customer’s data rate requirement increases. Recall the 
relationship in Equation (5.20b): 



If we do nothing to the system except increase the data rate R, the above expres¬ 
sion shows that the received F^/ZV,, would decrease, and in Figure 6.9, the operating 
point would move upwards from point D to, let us say, some point F. Now, envision 
“walking’’ down the vertical line to point F on the curve that represents coded 
modulation. Increasing the data rate has degraded the quality of the data. But, the 
use of error-correction coding brings back the same quality at the same power level 
{PrlN^^). The Ef,IN(i is reduced, but the code facilitates getting the same error 
probability with a lower Ef,IN(^. What price do we pay for getting this higher data 
rale or greater capacity? The same as before—increased bandwidth. 
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6.3.4.5 Trade-Off 4: Capacity versus Bandwidth 

Trade-ofl 4 is similar to trade-off 3 because both achieve increased capacity* 
A spread-spectrum multiple access technique, called code-division multiple access 
(CDMA) and described in Chapter 12, is one of the schemes used in cellular tele¬ 
phony. In CDMA, where users simultaneously share the same spectrum, each user 
is an interferer to each of the other users in the same cell or nearby cells. Hence, 
the capacity (maximum number of users) per cell is inversely proportional to 

(See Section 12.8.) In this application, a lowered results in a raised ca¬ 
pacity; the code achieves a reduction in each users power, which in turn allows for 
an increase in the number of users. Again, the cost is more bandwidth. But, in this 
case, the signal-bandwidth expansion due to the error-correcting code is small com¬ 
pared with the more significant spread-spectrum bandwidth expansion, and thus, 
there is no impact on the transmission bandw idth. 

In each of the above trade-off examples, a “tradilionar’ code involving redun¬ 
dant bits and faster signaling (for a real-time communication system) has been as¬ 
sumed; hence, in each case, the cost was expanded bandwidth. However, there 
exists an error-correcting technique, called trellis-coded modulation, that does not 
require faster signaling or expanded bandwidth for real-time systems. (This tech¬ 
nique is described in Section 9.10.) 


Example 6.2 Coded versus Uncoded Performiince 

Compare the message error probability for a communications link with and withoul 
the use of error-correction coding. Assume that the uncoded transmi.ssion characteris¬ 
tics are; BPSK modulation. Gaussian noise, = 43,776. data rale R = 4800 bils/s. 
For the coded case, also assume the use of a (15,11) error-correcting code that is capa¬ 
ble of correcting any single-error pattern within a block of 15 bits. Consider that the 
demodulator makes hard decisions and thus feeds the demodulated code bits directly 
to the decoder, which in turn outputs an estimate of the original message. 


Solution 

Following Equation (4.79). let /?„ = QS/lEi^jN,} and = Q\/2E^ INq be the uncoded 
and coded channel symbol error probabilities, respectively, w'here Et,INu is the bit 
energy per noi.se spectral density and E^/Nq is the code-bit energy per noise spectral 
density. 

Without coding 


N, % V R 


9.12 (9.6 dB) 


and 


Pu^Q 



2 ^ 

No 


C>(V18.24) = 1.02 X 10 


— ■> 


( 6 . 20 ) 


where the following approximation of Q(x) from Equation (3.44) was u.sed 


C?(.v) 


I 




exp 


lor .r > 3 
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Tlie probability that the uncoded message block will be received in error is 
1 minus the product of the probabilities that each bit will be delected correctly. Thus, 


n = 1 -(1 -p„f 

probability that all 
11 bits in uncoded 
block are correct 


= 1.12 X 10-' 

•-V-- 

probability that at 
least 1 bit out of 
II is In error 


(h.21) 


With coding 

Assuming a real-time communication system such that delay is unacceptable, the 
channel-symbol rate or code-bit rate K,. is 15/11 times the data bit rale: 

^ /?. = 48tK) X {f = 6545 bps 

and 



= 6.69(8.3 dB) 


Tire /:, /A/,) for each code hit is less than that for the data bit in the uncoded case 
because the channel-bit rate has increased, but the transmitter power is assumed to 
be fixed: 



^ (?(V13.38 = 


1.36 X lO"" 


( 6 . 22 ) 


It can be seen by comparing the results of Equation 6.20 with those of Equation 
6.22 that because redundancy was added, the channel bit-error probability has de¬ 
graded. More bits must be delected during the same time interval and with the same 
available power; the performance improvement due to the coding is not yet apparent. 
We now compute the coded message error rale using Equation 6.18: 


pc 


M 


;( = 15 


15 

/ 




rhe summation is started with / = 2, since the code corrects all single 
within a block of n = 15 bits. An approximation is obtained by using only the fir 
ol the summation. For p^, we use the value calculated in Equation 6.22: 



term 


1/2) ~ ( 6 - 23 ) 

By comparing the results of Equation (6.21) with (6.23), we can see that the probabil¬ 
ity of message error has improved by a factor of 58 due to the error-correcting code 
used in this example. This example illustrates the typical behavior of all such real-time 
communication systems using error-correction coding. Added redundancy means 
faster signaling, less energy per channel symbol, and more errors out of the demodula¬ 
tor. The benefits arise because the behavior of the decoder will (at reasonable values 
of E(y/N^^) more than compensate for the poor performance of the demodulator. 
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6.3.4.6 Code Performance at Low Values of 


The reader is urged to solve Problem 6.5, which is similar to Example 6.2. In 
pari (a) of Problem 6.5, where an of 14 dB is given, the result is a message- 

error performance improvement through the use of coding. However, in part (b) 
where the has been reduced to 10 dB, coding provides no improvement: in 

fact, there is a degradation. One might ask, Why does part (b) manifest a degrada¬ 


tion? After all, the same procedure is used lor applying the code in both parts ot 


■ - - . 

the problem. The answer can be seen in the coded-versus-uncoded pictorial shown 
in Figure 6.9. Even though Problem 6.5 deals with message-error probability, and 
Figure 6.9 displays bit-error probability, the following explanation still applies. In 
all such plots, there is a crossover between the curves (usually at some low value of 

The reason for such crossover (threshold) is that every code system has 
some fixed error-correcting capability. II there are more errors within a block than 
the code is capable of correcting, the system will perform poorly. Imagine that 

is continually reduced. What happens at the output of the demodulator? 
It makes more and more errors, rherefore. such a continual decrease in 
must evenluallv cause some threshold to be reached where the decoder becomes 


overwhelmed with errors. When that threshold is crossed, we can interpret ithe 
degraded performance as being caused by the redundant bits consuming 
energy but giving back nothing beneficial in return. Does it strike the reader 
as a paradox that operating in a region (low values of where one would best 

like to see an error-performance improvement, is where the code makes things 
worse? There is, however, a class of powerful codes called turbo codes that provide 
error-performance improvements at low values of the crossover point is 

lower for turbo codes compared with conventional codes. (These are treated in 

Section 8.4.) 


6.4 LINEAR BLOCK CODES 


Linear block codes (such as the one described in Example 6.2) are a class ol parity- 
check codes that can be characterized by the (/i. k) notation described earlier. The 
encoder transforms a block of k message digits (a message vector) into a longer 
block of n codeword digits (a code vector) constructed from a given alphabet of el¬ 
ements. When the alphabet consists of tw'o elements (0 and I), the code is a binary 
code comprising binary digits (bits). Our discussion of linear block codes is re¬ 
stricted to binary codes unless oiherw ise noted. 

The k-bit messages form 2^ distinct message sequences, relcrred to as k-tiiples 
(sequences of k digits). The //-bit blocks can form as many as 2” distinct sequences, 
referred to as n-tuples. The encoding procedure assigns to each of the 2^ message 
A'-tuplcs one of the 2" ^/-tuples. A block code represents a one-to-one assignment, 
whereby the 2^ message /c-luples are imuiuely mapped into a new set ot 2^ code¬ 
word //-tuples; the mapping can be accomplished via a look-up table. For linear 
codes, the mapping transformation is. of course linear. 
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6.4.1 Vector Spaces 


The set of all binary /j-tuples, is called a vector space over the binary field of 
two elements (0 and 1). The binary field has two operations, addition and multipli¬ 
cation, such that the results of all operations are in the same set of two elements. 
The arithmetic operations of addition and multiplication are defined by the con¬ 
ventions of the algebraic field [4]. For example, in a binary field, the rules of addi¬ 
tion and multiplication are as follows: 

Addition Multiplication 

0@0 = 0 0-0 = 0 

0 @ 1 = 1 0 - 1=0 

100=1 1-0 = 0 

101=0 1-1 = 1 

The addition operation, designated with the symbol 0, is the same modulo-2 
operation described in Section 2.9.3. The summation of binary n-tuples always 
entails modulo-2 addition. However, for notational simplicity the ordinary -i- sign 
will often be used. 

6.4.2 Vector Subspaces 


A subset 5 of the vector space V„ is called a subspace if the following two condi¬ 
tions arc met: 


1. The all-zeros vector is in S. 

2. The sum of any two vectors in S is also in S (known as the closure property). 


These properties are fundamental for the algebraic characterization of linear block 
codes. Suppose that V, and are two codewords (or code vectors) in an («, k) bi¬ 
nary block code. The code is said to be linear if, and only if (V, 0 V,) is also a code 
vector. A linear block code, then, is one in which vectors outside the subspace can¬ 
not be created by the addition of legitimate codewords (members of the subspace). 

For example, the vector space is totally populated by the following 2^ = 
sixteen 4-tuples: 


oooo om 0010 0011 oioo oioi oiio oiii 

KMK) 1001 1010 1011 1100 1101 1110 1111 

An example of a subset of that forms a subspace is 

0(K)0 0101 1010 nil 


It is easy to verify that the addition of any two vectors in the subspace can only 
yield one of the other members of the subspace. A set of 2* / 2 -tuples is called a lin¬ 
ear block code if, and only if, it is a subspace of the vector space V„ of all / 2 -tuples. 
Figure 6.10 illustrates, with a simple geometric analogy, the structure behind linear 
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2'^ n-tuples constitute 
the entire space Vn 



2* ri-tuples constitute 
the subspace of codewords 


Figure 6.10 Linear block-code structure. 

block codes. We can imagine the vector space comprising 2" w-tuples. Within 
this vector space there exists a subset of 2^ ^i-tuples making up a subspace. These 2* 
vectors or points, shown “sprinkled" among the more numerous 2” points, repre¬ 
sent the legitimate or allowable codeword assignments. A message is encoded into 
one of the 2* allowable code vectors and then transmitted. Because of noise in the 
channel, a perturbed version of the codeword (one of the other 2” vectors in the 
Az-tuple space) may be received. If the perturbed vector is not too unlike (not too 
distant from) the valid codeword, the decoder can decode the message correctly. 
The basic goals in choosing a particular code, similar to the goals in selecting a set 
of modulation waveforms, can be stated in the context of Figure 6.10 as follows: 




We strive for coding efficiency by packing the V„ space with as many code¬ 
words as possible. This is tantamount to saying that we only want to expend a 
small amount of redundancy (e.xcess bandwidth). 

We want the codewords to be as far apart from one another as possible, so 


• & 

may still be correctly decoded, with a high probability 


transmission 


6.4.3 A (6, 3) Linear Block Code Example 


Examine the following coding assignment that describes a (6. 3) code. There are 
2* = 2- = 8 message vectors, and therefore eight codewords. There are 2* = 2^ = 
sixty-four 6-tuples in the vector space: 
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It is easy to check that the eight codewords shown in Table 6.1 form a sub¬ 
space of (the all-zeros vector is present, and the sum of any two codewords 
yields another codeword member of the subspace). I’herefore, these codewords 
represent a linear block code, as defined in Section 6.4.2. A natural question to ask 
is How was the codeword-to-message assignment chosen for this (6, 3) code? A 
unique assignment for a particular {n, k) code does not exist; however, neither is 
there complete freedom of choice. In Section 6.6,3, we examine the requirements 
and constraints that drive a code design. 


TABLE 6.1 Assignment of Codewords to Messages 


Message vector Codeword 


0 0 0 
1 00 
010 
1 1 0 
0 0 I 

1 0 I 
0 1 1 
111 


0 i.) 0 0 00 
1 1010 0 
0 t 10 10 
10 11 10 
10 10 0 1 
0 1110 1 
1 10 011 
0 0 0 1 1 1 


6.4.4 Generator Matrix 

If k is large, a table look-up implementation ot the encoder becomes prohibitive. 

For a (127, 92) code there are or approximately 5 x code vectors. If the 

encoding procedure consists of a simple look-up table, imagine the size of the 

memory necessar>' to contain such a large number of codewords. Fortunately, it is 

possible to reduce complexity by generating the required codewords as needed, 
instead of storing them. 

Since a set of codewords that forms a linear block code is a k-dimensional 
subspace of the n-dimensional binary vector space (k < n). it is always possible to 
find a set of n-tuples, fewer than 2^ that can generate all the 2* codewords of the 
subspace. The generating set of vectors is said to span the subspace. TTie smallest 
linearly independent set that spans the subspace is called a basis of the subspace, 
and the number of vectors in this basis set is the dimension of the subspace. Any 
basis set of k linearly independent n-tuples V,, V 2 ,... , can be used to generate 
the required linear block code vectors, since each code vector is a linear combina¬ 
tion of Vj, V 2 , ... , Vji. That is, each of the set of 2* codewords {U} can be de¬ 
scribed by 

U= -f- m2\2 

where m, = (0 or 1) are the message digits and / = 1,..., 

In general, we can define a generator matrix by the following kxn array: 
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Vi V’li Vi2 

^2 _ V ^21 ^22 

• « 

vj Lv'a-i Va2 

Code vectors, by convention, are usually designated as row vectors. Thus, the mes¬ 
sage m, a sequence of k message bits, is shown below as a row vector (1 x /c matrix 

having one row and k columns): 

m = mj, mj, 

The generation of the codeword U is written in matrix notation as the product of ni 
and G, and we write 

U = mG (6.25) 

where, in general, the matrix multiplication C = AB is performed in the usual way 
by using the rule 





where A is an / x ai matrix, B is an n x rn matrix, and the result C is an / x rn matrix. 
For the example introduced in the preceding section, we can fashion a generator 

matrix as 

'Vi] ri 1 0 1 0 o‘ 

G = V 2 = 0 1 1 0 1 0 (6.26) 

-V 3 J Ll 0 1 0 0 1. 

where V^, V 2 , and V 3 are three linearly independent vectors (a subset of the eight 
code vectors) that can generate all the code vectors. Notice that the sum of any two 
generating vectors does not yield any of the other generating vectors (opposite of 
closure). Let us generate the codeword U 4 for the fourth message vector 1 1 0 in 
Table 6.1, using the generator matrix of Equation (6.26): 

U4 = [l 1 0] V 2 = l-Vi + 1-V2 + 0-V3 

LV 3 . 

= 1 1 0 10 0 + 0 1 10 1 0 + 0 0 0 0 0 0 

= 10 1110 (codeword for the message vector 110) 

Thus, the code vector corresponding to a message vector is a linear combination of 
the rows of G. Since the code is totally defined by G, the encoder need only store 
the k rows of G instead of the total 2^" vectors of the code. For this example, notice 
that the generator array of dimension 3 x 6 in Equation (6.26) replaces the original 
codeword array of dimension 8 x 6 in Table 6.1, representing a reduction in system 

complexity. 
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6.4.5 Systematic Linear Block Codes 


A systematic (n, k) linear block code is a mapping from a ^-dimensional message 
vector to an /i-dimensional codeword in such a way that part of the sequence gener¬ 
ated coincides with the k message digits. The remaining (n - k) digits are parity 
digits. A systematic linear block code will have a generator matrix of the form 



Pll 

P\2 

••• Phin 

Pll 

• 

P 22 

''' P2in 

1 

4 

Pk2 

Pk.(n 


k) 1 0 0 

0 1 ••• 0 

;t) 0 0 ••• 1 



where P is the parity array portion of the generator matrix = (0 or 1), and 1;^ is 
the k X k identity matrix (ones on the main diagonal and zeros elsewhere). Notice 
that with this systematic generator, the encoding complexity is further reduced 
since it is not necessary to store the identity matrix portion of the array. By com- 


bining Equations (6.26) and (6.27), each codeword is expressed as 




>11 

P\2 

P\,(n-k) 1 0 

0 

1 ^ 9 

= [m,,m 2 ,... ,m^] X 

7^21 

t 

• 

t 

P 22 

Plin-k) 0 1 

0 

4 

4 

4 



-Pk\ 

Pk2 

Pk^n-k) 0 0 

1 

where 






w, = 

^ m,/?,, + m 2 P 2 i+ 

+ nikPki for/ = 

= !,...,(« - k) 



^i-n + k 


for / = 

= {n — k + 1 ),..., /? 



Given the message /c-tuple 


m = mj, m2,..., m^ 


and the general code vector «-tuple 



M], M2» 


• • 


u 


n 


the systematic code vector can be expressed as 


where 




P\> Pl^ ' " y P n-ky ^\y ' y ^ k 


-V- 

parity bits 




message bits 


(6.28) 
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(6.29) 


Pi = "»lPl2 ■*" "*2P22 

Pn-k = f^hPUr,-k) +m2P2i,n-k) + + f^HPLK„-k) 


Systematic codewords are sometimes written so that the message bits occupy 
the left-hand portion of the codeword and the parity bits occupy the right-hand 
portion. This reordering has no ellect on the error detection or error correction 

properties of the code, and will not be considered further. 

For the (6, 3) code example in Section 6.4.3, the codewords are described as 

follows: 



= mi + mj. 



m, + mi, T 




U2 




ii 





(6.31) 


Equation (6.31) provides some insight into the structure of linear block codes. 
We see that the redundant digits are produced in a variety of ways. The first parity 
bit is the sum of the first and third message bits; the second parity bit is the sum of 
the first and second message bits; and the third parity bit is the sum of the second 
and third message bits. Intuition tells us that such structure, compared with 
single-parity checks or simple digit-repeat procedures, may provide greater ability 

to detect and correct errors. 


6.4.6 Parity-Check Matrix 


Let us define a matrix H, called the parity-check matrix, that will enable us to 
decode the received vectors. For each {k x n) generator matrix G, there exists an 
in - k) X n matrix H, such that the rows of G are orthogonal to the rows of H; that 
is, GH^ = 0, where is the transpose of H, and 0 is a k x [n - k) all-zeros matrix. 

is an n x (^? - k) matrix whose rows are the columns of H and whose columns 
are the rows of H. To fulfill the orthogonality requirements for a systematic code, 
the components of the H matrix are written as 





(6.32) 


Hence, the matrix is written as 



(6.3.3a) 
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1 

0 

0 

0 

• 

• 

1 

0 

• 

0 

0 

1 

Pu 

Pu 

Phin-k) 

P2\ 

• 

• 

P22 

P2Xn-k) 

• 

_ Pkl 

Pk2 

Pkin-k) 


(6.33b) 


It is easy to verify th 
theH^ matrix, yields 


of each codeword U, generated by G and 


UH''= Pi + Pi, P2 + P2, .... P„-t + P„_J = 0 

where the parity bits P\,p 2 ,... are defined in Equation (6.29). Thus, once the 
parity-check matrix H is constructed to fulfill the foregoing orthogonality require¬ 
ments, we can use it to test whether a received vector is a valid member of the 
codeword set. U is a codeword generated by matrix G if, and only if UH^= 0. 


6.4.7 Syndrome Testing 


Let r = rj, r 2 , ... , be a received vector (one of 2” ^-tuples) resulting from the 

transmission of U = Mj, Uj, ... ^ u„ (one of 2^ n-tuples). We can therefore describe 
r as 


r = U + e (6.34) 

where e = e,, ^ 2 , ... , is an error vector or error pattern introduced by the 
channel. There are a total of 2" — 1 potential nonzero error patterns in the space of 
2” n-tuples. The syndrome of r is defined as 

S = rH^ (6.35). 

The syndrome is the result of a parity check performed on r to determine whether r 
is a valid member of the codeword set. If, in fact, r is a member, the syndrome S has 
a value 0. If r contains detectable errors, the syndrome has some nonzero value. If r 
contains correctable errors, the syndrome (like the symptom of an illness) has some 
nonzero value that can earmark the particular error pattern. The decoder, depend¬ 
ing upon whether it has been implemented to perform EEC or ARQ, will then take 
actions to locate the errors and correct them (EEC), or else it will request a retrans¬ 
mission (ARQ). Combining Equations (6.34) and (6.35), the syndrome of r is seen 
to be 


S = (U+ e)H^ 

= UH^+ eH^ 

However, UH^ = 0 for all members of the codeword set. Therefore, 

S = eH^ 



(6.37) 
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The foregoing development, starting with Equation (6.34) and terminating with 
Equation (6.37), is evidence that the syndrome test, whether performed on either a 
corrupted code vector or on the error pattern that caused it, yields the same syn¬ 
drome. An important property of linear block codes, fundamental to the decoding 
process, is that the mapping between correctable error patterns and syndromes is 

one to one. 

It is interesting to note the following two required properties of the parity- 
check matrix. 

1. No column of H can be all zeros, or else an error in the corresponding code¬ 
word position would not affect the syndrome and would be undetectable. 

2. All columns of H must be unique. If two columns of H were identical, errors 
in these two corresponding codeword positions would be indistinguishable. 

Example 6.3 Syndrome Test 

Suppose that codeword U = 101110 from the example in Section 6.4.3 is transmitted 
and the vector r = 0 0 1 1 1 0 is received; that is, the leftmost bit is received in error. 
Find the syndrome vector value S = rH^ and verify that it is equal to eH^ 

Solution 

S = rH^ 


= 001 1 10 ] 


= [1, 1 + 1, 1 + 1] = [1 0 0] (syndrome of corrupted code vector) 

Next, we verify that the syndrome of the corrupted code vector is the same as 
the syndrome of the error pattern that caused the error: 

S = eH^ = [1 0 0 0 0 0]H^ = [1 0 0] (syndrome of error pattern) 

6.4.8 Error Correction 

We have detected a single error and have shown that the syndrome test performed 
on either the corrupted codeword, or on the error pattern that caused it, yields the 
same syndrome. This should be a clue that we not only can detect the error, but 
since there is a one-to-one correspondence between correctable error patterns and 
syndromes, we can correct such error patterns. Let us arrange the 2” w-tuples that 
represent possible received vectors in an array, called the standard array, such that 
the first row contains all the codewords, starting with the all-zeros codeword, and 
the first column contains all the correctable error patterns. Recall from the basic 
properties of linear codes (see Section 6.4.2) that the all-zeros vector must be a 
member of the codeword set. Each row, called a coset, consists of an error pattern 
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in the first column, called the coset leader, followed by the codewords perturbed by 
that error pattern. The standard array format for an {n, k) code is as follows: 



U2 

• • • u, 

V^k 

^2 

U2 + e2 

U, e2 

^2*+ e2 

Cl 

• 

• 

U2 + e 3 

♦ 

• 

U, + e3 

• 

• 

U2*'+ e 3 

• 

• 

• 

♦ 

U2 + e, 

• 

♦ 

• 

+ e, 

• 

♦ 

U2* -l-e^ 

• 

• 

U2 “1“ 62"”^’ 

• 

U, + e2"“^' ••• 

U2^+ 62"”* 



Note that codeword Uj, the all-zeros codeword, plays two roles. It is one of the code¬ 
words, and it can also be thought of as the error pattern e, —the pattern that repre¬ 
sents no error, such that r = U. The array contains all 2" Aj-tuples in the space V„. Each 
/i-tuple appears in only one location—none are missing, and none are replicated. 
Each coset consists of 2* Az-tuples. Therefore, there are (2"/2^) = 2""* cosets. 

The decoding algorithm calls for replacing a corrupted vector (any n-tuple ex¬ 
cluding those in the first row) with a valid codeword from the top of the column 
containing the corrupted vector. Suppose that a codeword U, (/ = 1, ... , 2*) is 
transmitted over a noisy channel, resulting in a received (corrupted) vector U, -i- e,. 
If the error pattern e. caused by the channel is a coset leader, where the index 

j ' 

y = 1, ... , 2”~ , the received vector will be decoded correctly into the transmitted 
codeword U,. If the error pattern is not a coset leader, then an erroneous decoding 
will result. 


6.4.8.1 The Syndrome of a Coset 

If ey is the coset leader or error pattern of the yth coset, then U, -i- e, is an 
/z-tuple in this coset. The syndrome of this n-tuple can be written 

S = {Vi + e^)H^= U,H^+ e,H^ 

Since U, is a code vector, U,H^ = 0, and we can write, as in Equation (6.37), 

• S = (U,+e,)H^= e,H'’ (6.39) 

The name coset is short for “a set of numbers having a common feature.” What do 
the members of any given row (coset) have in common? From Equation (6.39) it is 
clear that each member of a coset has the same syndrome. TTie svndrome for each 
coset is different from that of any other coset in the code; it is the syndrome that is 
used to estimate the error pattern. 

6.4.8.2 Error Correction Decoding 

The procedure for error correction decoding proceeds as follows: 

1. Calculate the syndrome of r using S = rH^. 

2. Locate the coset leader (error pattern) e^, whose syndrome equals rH^. 


6.4 


Linear Block Codes 


337 



3. This error pattern is assumed to be the corruption caused by the channel. 

4. The corrected received vector, or codeword, is identified as U = r + ej. We can 
say that we retrieve the valid codeword by subtracting out the identified 
error; in modulo-2 arithmetic, the operation of subtraction is identical to that 
of addition. 

6.4.8.3 Locating the Error Pattern 

Returning to the example of Section 6.4.3, we arrange the 2^ = sixty-four 
6-tuples in a standard array as shown in Figure 6.11. The valid codewords are the 
eight vectors in the first row, and the correctable error patterns are the seven 
nonzero coset leaders in the first column. Notice that all 1-bit error patterns are cor¬ 
rectable. Also notice that after exhausting all 1-bit error patterns, there remains 
some error-correcting capability since we have not yet accounted for all sixty-four 
6-tuples. There is one unassigned coset leader; therefore, there remains the capa¬ 
bility of correcting one additional error pattern. We have the flexibility of choosing 
this error pattern to be any of the Ai-tuples in the remaining coset. In Figure 6.11 
this final correctable error pattern is chosen, somewhat arbitrarily, to be the 2-bit 
error pattern 0 1 0 0 0 1. Decoding will be correct if, and only if, the error pattern 
caused by the channel is one of the coset leaders. 


000000 

110100 

011010 

101110 

101001 

011101 

110011 

000111 

000001 

110101 

011011 

101111 

101000 

011100 

110010 

000110 

000010 

110110 

011000 

101100 

101011 

011111 

110001 

000101 

000100 

110000 

011110 

101010 

101101 

011001 

110111 

000011 

001000 

111100 

010010 

100110 

100001 

010101 

111011 

001111 

010000 

100100 

001010 

111110 

111001 

001101 

100011 

010111 

100000 

010100 

111010 

001110 

001001 

111101 

010011 

100111 

010001 

100101 

001011 

mill 

111000 

001100 

100010 

010110 


Figure 6.11 Example of a standard array for a (6, 3) code. 


We now determine the syndrome corresponding to each of the correctable 
error sequences by computing for each coset leader, as follows: 

1 0 o' 

0 1 0 

0 0 1 

1 1 0 

0 1 1 

1 0 1 



338 


Channel Coding: Part 1 Chap. 6 


The results are listed in Table 6.2. Since each syndrome in the table is unique, the 
decoder can identify the^rror pattern e to which it corresponds. 


TABLE 6.2 Syndrome Look-Up Table 


Error^attern 

Svndrome 

V 

000000 

000 

00000 1 

101 

0000 1 0 

01 1 

000 100 

1 10 

0 0 10 0 0 

00 1 

0 1 0 0 0 0 

0 1 0 

1 0 0 0 0 0 

1 0 0 

010001 

111 


6.4.K.4 Error Correction Example 

As outlined in Section 6.4.8.2 we receive the vector r and calculate its syn¬ 
drome using S = rH^ We then use the syndrome look-up table (Table 6.2), devel¬ 
oped in the preceding section, to find the corresponding error pattern. This error 
pattern is an estimate of the error, and we denote it e. The decoder then adds e to r 
to obtain an estimate of the transmitted codeword U: 

ij=r-Le = (U+e) + e = U-f(e-Le) (6.40) 

If the estimated error pattern is the same as the actual error pattern, that is, if e = e, 
then the estimate U is equal to the transmitted codeword U. On the other hand, if 
the error estimate is incorrect, the decoder will estimate a codeword that was not 
transmitted, and we have an undetectable decoding error. 

Example 6.4 Error Correction 

Assume that codeword LJ = 101110, from the Section 6.4.3 example, is transmitted, 
and the vector r = 0 0 1 1 1 0 is received. Show how a decoder, using the Table 6.2 
syndrome look-up table, can correct the error. 

Solution 

The syndrome of r is computed: 

S = [0 0 1 1 1 0]H^= [1 0 0] 

Using Table 6.2, the error pattern corresponding to the syndrome above is estimated 
to be 


e = 1 0 0 0 0 0 

The corrected vector is then estimated by 

A 

U= r + e 

= 001 1 lO-HlOOOOO 
= 10 1110 
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Since the estimated error pattern is the actual error pattern in this example, the error 
correction procedure yields U = U. 

Notice that the process of decoding a corrupted codeword by first delecting and 
then correcting an error can be compared to a familiar medical analogy. A patient 
(potentially corrupted codeword) enters a medical facility (decoder). The examining 
physician performs diagnostic testing (multiplies by H^) in order to find a symptom 
(syndrome). Imagine that the physician finds characteristic spots on the patient's 
x-rays. An experienced physician would immediately recognize the correspondence 
between the symptom and the disease (error pattern) tuberculosis. A novice physician 
might have to refer to a medical handbook (Table 6.2) to associate the symptom or 
syndrome with the disease or error pattern. The final step is to provide the proper 
medication that removes the disease, as seen in Equation (6.40). In the context of 
binary codes and the medical analogy, Equation (6.40) reveals an unusual type of 
medicine practiced here. The patient is cured by reapplying the original disease. 


6.4.9 Decoder Implementation 


When the code is short as in the case of the (6, 3) code described in the previous 
sections, the decoder can be implemented with simple circuitry. Consider the steps 
that the decoder must take: (1) calculate the syndrome, (2) locate the error pattern, 
and (3) perform modulo-2 addition of the error pattern and the received vector 
(which removes the error). In Example 6.4, we started with a corrupted vector and 
saw how these steps yielded the corrected codeword. Now, consider the circuit in 
Figure 6.12, made up of exclusive-OR gates and AND gates that can accomplish 
the same result for any single-error pattern in the (6, 3) code. From Table 6.2 and 
Equation 6.39, we can write an expression for each of the syndrome digits in terms 
of the received codeword digits as 




5, = rj + r4 -f 

52 = r2 ■*" '■4 '■5 

^3 = '■3 + ^5 + ''6 


We use these syndrome expressions for wiring up the circuit in Figure 6.12. The 
exclusive-OR gate is the same operation as modulo-2 arithmetic and hence uses the 
same symbol. A small circle at the termination of any line entering the AND gate 
indicates the logic COMPLEMENT of the signal. 
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Figure 6.12 Implementation of the (6, 3) decoder. 

The corrupted signal enters the decoder at two places simultaneously. At the 
upper part of the circuit, the syndrome is computed, and at the lower part, that 
syndrome is transformed to its corresponding error pattern. The error is removed 
by adding it back to the received vector yielding the corrected codeword. 

Note that, for tutorial reasons. Figure 6.12 has been drawn to emphasize the 
algebraic decoding steps—calculation of syndrome, error pattern, and corrected 
output. In the real world, an {n, k) code is usually configured in systematic form. 
The decoder would not need to deliver the entire codeword; its output would con¬ 
sist of the data bits only. Hence, the Figure 6.12 circuitry becomes simplified by 
eliminating the gates that are shown with shading. For longer codes, such an imple¬ 
mentation is very complex, and the preferred decoding techniques conserve cir¬ 
cuitry by using a sequential approach instead of this parallel method [4]. It is also 
important to emphasize that Figure 6.12 has been configured to only detect and 

correct single-error patterns for the (6, 3) code. Error control for a double-error 
pattern would require additional circuitry. 

6.4,9.1 Vector Notation 

Codewords, error patterns, received vectors, and syndromes have been de¬ 
noted by the vectors U, e, r, and S respectively. For notational simplicity, an index 
to denote a particular vector has generally been left off. However, to be precise, 
each of these vectors U, e, r, and S is one of a set having the general form 
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Consider the range of the indices j and i in the context of the (6, 3) code in Table 
6.1. For the codeword Uy, the index y = 1,... , 2*^ indicates that there are 2^ = 8 dis¬ 
tinct codewords, and the index i = l,... ,n indicates that each codeword is made up 
of A7 = 6 bits. For a correctable error pattern Cy, the index / = 1,... , 2"'* indicates 
that there are 2^ = 8 coset leaders (7 nonzero correctable error patterns), and the 
index i = I,... ,n indicates that each error pattern is made up of n = 6 bits. For the 
received vector r^, the index y = 1, ... , 2” indicates that there are 2^ = 64 possible 
A7-tuples that can be received, and the index i=l,... ,n indicates that each received 
^7-tuple is made up of « = 6 bits. Finally, for the syndrome Sy, the index y = 1,... , 
2'’"^ indicates that there are 2^ = 8 distinct syndrome vectors, and the index / = 1, 
... ,n - k indicates that each syndrome is made up of n- k = 3 bits. In this chapter, 
the index is often dropped, and the vectors Uy, Cy, ry, and Sy are denoted as U, e, r, 
and S, respectively. The reader must be aware that for these vectors, an index is 
always inferred, even when it has been left off for notational simplicity. 


6.5 ERROR-DETECTING AND CORRECTING CAPABILITY 

6.5.1 Weight and Distance of Binary Vectors 

It should be clear that not all error patterns can be correctly decoded. The error 
correction capability of a code will be investigated by first defining its structure. 
The Hamming weight »v(U) of a codeword U is defined to be the number of 
nonzero elements in U. For a binary vector this is equivalent to the number of ones 
in the vector. For example, if U = 1 0 0 1 0 1 1 0 1, then w(U) = 5. The Hamming 
distance between two codewords U and V, denoted d(\J, V), is defined to be the 
number of elements in which they differ—for example, 

U = 1 0 0 1 0 1 10 1 
V = 0 1 1 1 10 10 0 
d{V, V) = 6 

By the properties of modulo-2 addition, we note that the sum of two binary vectors 
is another vector whose binary ones are located in those positions in which the two 
vectors differ—for example 

U+V=l 1 10 1 1001 

Thus, we observe that the Hamming distance between two codewords is equal to 
the Hamming weight of their sum: that is, r/(U, V) = w(U -i- V). Also we see that the 
Hamming weight of a codeword is equal to its Hamming distance from the all-zeros 
vector. 
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6.5.2 Minimum Distance of a Linear Code 


Consider the set of distances between all pairs of codewords in the space V„. The 
smallest member of the set is the minimum distance of the code and is denoted 
Why do you suppose we have an interest in the minimum distance; why not the 
maximum distance? The minimum distance, like the weakest link in a chain, gives 
us a measure of the code’s minimum capability and therefore characterizes the 
code’s strength. 

As discussed earlier, the sum of any two codewords yields another codeword 
member of the subspace. This property of linear codes is stated simply as: If U and 
V are codewords, then W = U + V must also be a codeword. Hence the distance be¬ 
tween two codewords is equal to the weight of a third codeword; that is, d{lJ, V) = 
w(U + V) = H'(W). Thus the minimum distance of a linear code can be ascertained 
without examining the distance between all combinations of codeword pairs. We 
only need to examine the weight of each codeword (excluding the all-zeros code¬ 
word) in the subspace; the minimum weight corresponds to the minimum distance, 
djnin^ Equivalently, corresponds to the smallest of the set of distances between 
the all-zeros codeword and all the other codewords. 

6.5.3 Error Detection and Correction 

The task of the decoder, having received the vector r, is to estimate the transmitted 
codeword U,. The optimal decoder strategy can be expressed in terms of the maxi¬ 
mum likelihood algorithm (see Appendix B) as follows: Decide in favor of U,- if 

P(rm) = max P(r|Uy) (6.41) 

over all U, 

Since for the binary symmetric channel (BSC), the likelihood of U, with respect to r 
is inversely proportional to the distance between r and U„ we can write: Decide in 
favor of U/ if 


d(i, U,) = min U) (6-42) 

over all ^ 

In other words, the decoder determines the distance between r and each of the 
possible transmitted codewords Uy, and selects as most likely a Uj for which 

d{r, U,) < d{T, Uy) for /,y = 1,..., A/ and i ^ j (6.43) 

where A/ = 2* is the size of the codeword set. If the minimum is not unique, the 
choice between minimum distance codewords is arbitrary. Distance metrics are 
treated further in Chapter 7. 

In Figure 6.13 the distance between two codewords U and V is shown using a 
number line calibrated in Hamming distance. Each black dot represents a cor¬ 
rupted codeword. Figure 6.13a illustrates the reception of vector Fj, which is dis¬ 
tance 1 from U and distance 4 from V. An error-correcting decoder, following the 
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Region 1 


Decision 

line 


Region 2 




Figure 6.13 Error correction and detection 
capability, (a) Received vector r^. (b) Re- 
(c) ceived vector r 2 . (c) Received vector r 3 . 


maximum likelihood strategy, will select U upon receiving Ti. If F] had been the 
result of a 1-bit corruption to the transmitted code vector U, the decoder has suc¬ 
cessfully corrected the error. But if r, had been the result of a 4-bit corruption to 
the transmitted code vector V, the result is a decoding error. Similarly, a double 
error in transmission of U might result in the received vector r 2 , which is distance 2 
from U and distance 3 from V, as shown in Figure 6.13b. Here, too, the decoder 
will select U upon receiving r 2 . A triple error in transmission of U might result in a 
received vector r, that is distance 3 from U and distance 2 from V, as shown in 
Figure 6.13c. Here the decoder will select V upon receiving r^, and will have made 
an error in decoding. From Figure 6.13 it should be clear that if error detection and 
not correction is the task, a corrupted vector—characterized by a black dot and 
representing a 1-bit, 2-bit, 3-bit, or 4-bit error—can be detected. However, five 
errors in transmission might result in codeword V being received when codeword 
U was actually transmitted; such as error would be undetectable. 

From Figure 6.13 w'e can see that the error-detecting and error-correcting ca¬ 
pabilities of a code are related to the minimum distance between codewords. The 
decision line in the figure serves the same purpose in the process of decoding as it 
does in demodulation, to define the decision regions. In the Figure 6.13 example, 
the decision criterion of choosing U if r falls in region 1, and choosing V if r falls in 
region 2, illustrates that such a code (with d^^J^ = 5) can correct two errors. In gen¬ 
eral, the error-correcting capahility t of a code is defined as the maximum number 
of guaranteed correctable errors per codeword, and is written [4] 
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where lx] means the largest integer not to exceed .v. Often, a code that corrects all 
possible sequences of t or fewer errors can also correct certain sequences of r + 1 er¬ 
rors. ^rhis can be seen in Figure 6.11. In this example = 3, and thus from Equa¬ 
tion (6.44), we can see that a/I t = ] bit-error patterns are correctable. Also, a single 
? -I- 1 or 2-bit error pattern is correctable. In general, a f-error-correcting (n, A) 
linear code is capable of correcting a total of 2" " error patterns. If a r-error- 
correcting block code is used strictly for error correction on a binary symmetric ' 
channel (BSC) with transition probability p, the message-error probability, P^, 
that the decoder commits an erroneous decoding, and that the ^ 7 -bit block is in 
error, can be calculated by using Equation (6.18) as an upper bound: 



The bound becomes an equality when the decoder corrects all combinations of er¬ 
rors up to and including t errors, but no combinations of errors greater than t. Such 
decoders are called /mounded distance decoders. The decoded bit-error probability, 

F/j, depends on the particular code and decoder. It can be expressed [5] by the 
following approximation: 








A block code needs to detect errors prior to correcting them. Or, it may be 
used for error-detection only. It should be clear from Figure 6.13 that any received 
vector characterized by a black dot (a corrupted codeword) can be identified as an 


error. I’hercfore, the error-detecting capability is defined in terms of d 


min 



d 


mm 




A block code with minimum distance guarantees that all error patterns of 
^min ~ 1 few'cr errors can be. detected. Such a code is also capable of detecting a 
large fraction of error patterns with d^^ or more errors. In fact, an (//, A) code is ca¬ 
pable of detecting 2” — 2^ error patterns of length n. The reasoning is as follows. 
There are a total of 2" - I possible nonzero error patterns in the space of 2" 
A7-tuples. Even the bit pattern of a valid codeword represents a potential error pat¬ 
tern. Thus there are 2^ - I error patterns that are identical to the 2^ - 1 nonzero 

a n y o f these 2^ - 1 error patterns occurs, it alters the transmitted 
codeword U, into another codeword U,. Thus will be received and its syndrome 
is zero. The decoder accepts Uy as the transmitted codew'ord and thereby commits 
an incorrect decoding. Therelore, there are 2* — 1 undetectable error patterns. If 
the error pattern is not identical to one of the 2* codewords, the syndrome test on 
the received vector r yields a nonzero syndrome, and the error is detected. There¬ 
fore, there are exactly T - 2^ detectable error patterns. For large /r, where 2* « 2”, 
only a small fraction of error patterns are undetected. 
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6.5.3.1 Codeword Weight Distribution 

Let Af be the number of codewords of weight j within an {n, k) linear code. 
The numbers /Iq, /t j, ... , are called the weight distrihution of the code. If the 
code is used only for error detection, on a BSC, the probability that the decoder 
does not detect an error can be computed from the weight distribution of the 
code [5] as 

>'nd = 2 V - P (6-48) 

where p is the transition probability of the BSC. If the minimum distance of the 
code is the values of A i to _ i are zero. 

Example 6^ Probability of an Undetected Error in an Error Detecting Code 

Consider that the (6, 3) code, given in Section 6.4.3, is used only for error detection. 
Calculate the probability of an undetected error if the channel is a BSC and the transi¬ 
tion probability is 10"^. 

Solution 

The weight distribution of this code is y4o = 1, /tj = >42 = = 4, /t 4 = 3, A^ = 0, >4^ = 0. 

Therefore, we can write, using Equation (6.48), 

Pnd = 4/r’(l - pf + 3p'(l - p)2 
For p = 10“^, the probability of an undetected error is 3.9 x 10“^ 

6.5.3.2 Simultaneous Error Correction and Detection 

It is possible to trade correction capability from the maximum guaranteed (/) 
where t is defined in Equation (6.44), for the ability to simultaneously detect a class 
of errors. A code can be used for the simultaneous correction of a errors and detec¬ 
tion of p errors, where p > a, provided that its minimum distance is [4] 

dmin ^ a + p + 1 (6.49) 

When t or fewer errors occur, the code is capable of delecting and correcting them. 
When more than t but fewer than e - 1 -1 errors occur, where e is defined in Equation 
(6.47), the code is capable of detecting their presence but correcting only a subset 
of them. For example, a code with = 7 can be used to simultaneously detect and 
correct in any one of the following ways: 

Detect (P) Correct (a) 

3 3 

4 2 

5 1 

6 0 

Note that correction implies prior detection. For the above example, when there 
are three errors, all of them can be detected and corrected. When there are five er¬ 
rors, all of them can be detected but only a subset of them (one) can be corrected. 
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6.5.4 Visualization of a 6-Tuple Space 


Figure 6.14 is a visualization of the eight codewords from the example of Section 
6.4.3. The codewords are generated from linear combinations of the three indepen¬ 
dent 6-tuples in Equation (6.26); the codewords form a three-dimensional sub¬ 
space. The figure shows such a subspacc completely occupied by the eight 
codewords (large black circles); the coordinates of the subspace have purposely 
been drawn to emphasize their nonorthogonality. Figure 6.14 is an attempt to illus¬ 
trate the entire space, containing sixty-four 6-tuples, even though there is no 
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Figure 6.14 Example of eight codewords in a 6-tuple space. 
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precise way to draw or construct such a model. Spherical layers or shells are shown 
around each codeword. Each of the nonintersecting inner layers is a Hamming dis¬ 
tance of 1 from its associated codeword; each outer layer is a Hamming distance of 
2 from its codeword. Larger distances are not useful in this example. Fe^r each 
codeword, the two layers shown are occupied by perturbed codewords. There are 
six such points on each inner sphere (a total of 48 points), representing the six pos¬ 
sible 1-bit error-perturbed vectors associated with each codeword. These 1-bit per¬ 
turbed codewords are distinct in the sense that they can best be associated with 
only one codeword, and therefore can be corrected. As is seen from the standard 


array of Figure 6,11, there is also one 2-bil error pattern that can be corrected. 



codeword, but only one of them, in our example the 0 10 0 0 1 error pattern, can be 
corrected. The other fourteen 2-bit error patterns yield vectors that cannot be 
uniquely identified with just one codeword; these noncorrectable error patterns 
yield vectors that are equivalent to the error-perturbed vectors of two or more 
codewords. In the figure, all correctable (fifty-six) 1- and 2-bit error-perturbed 
codewords are shown as small black circles. Perturbed codewords that cannot be 
corrected are shown as small clear circles. 

Figure 6.14 is useful for visualizing the properties of a class of codes known as 
perfect codes. A /-error-correcting code is called a perfect code if its standard array 
has all the error patterns of t and fewer errors and no others as cosel leaders (no 
residual error-correcting capacity). In terms of Figure 6.14, a r-error-correcting per¬ 
fect code is one that can, with maximuni likelihood decoding, correct all perturbed 
codewords occupying a shell at Hammiyig distance r or less from its originating 
codeword, and cannot correct any perturbed vectors occupying shells at distances 
greater than /. 

Figure 6.14 is also useful for understanding the basic goal in the search for 
good codes. We would like for the space to be filled with as many codewords as 
possible (efficient utilization of the added redundancy), and we would also like 
these codewords to be as far away from one another as possible. Obviously, these 
goals conflict. 


6.5.5 Erasure Correction 


A receiver may be designed to declare a symbol erased when it is received ambigu¬ 
ously or when the receiver recognizes the presence of interference or a transient 
malfunction. Such a channel has an input alphabet of size Q and an output alpha¬ 
bet of size Q + 1; the extra output symbol is called an erasure flag, or simply an era¬ 
sure, When a demodulator makes a symbol error, two parameters are needed to 
correct that error, its location and its correct symbol value. In the case of binary 
symbols, this reduces to needing only the error location. However, if the demodula¬ 
tor declares a symbol erased, although the correct symbol value is not known, the 
symbol location is known, and for this reason, the decoding of erased codewords 
can be simpler than error correcting. An error control code can be used to correct 
erasures or to correct errors and erasures simultaneously. If the code has minimum 
distance any pattern of p or fewer erasures can be corrected if [6| 
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(6.50) 


^^min — p 1 

Assume for the moment that no errors occur outside the erasure positions. The ad¬ 
vantage of correcting by means of erasures is exprc.ssed quantitatively as follows: If 
a code has a minimum distance then from Equation (6.50), - 1 erasures 

can be reconstituted. Since the number of errors that can be corrected without 
erasure information is - l)/2 at most, from Equation (6.44), the advantage of 
correcting by means of era.sures is clear. Further, any pattern of a errors and 
7 erasures can be corrected simultaneously if [ 6 ] 


^ 2a -t- 7 1 (6.51) 

Simultaneous erasure correction and error correction can he accomplished in 
the following way. First, the 7 -erased positions are replaced with zeros and the re¬ 
sulting codeword is decoded normally. Next, the 7 -erascd positions are replaced 
with ones, and the decoding operation is repeated on this version of the codeword. 
Of the two codewords obtained (one with erasures replaced by zeros, and the other 
with erasures replaced by ones) the one coiTcsponding to the smallest number 
of errors corrected outside the 7 -crased positions is selected. This technique will 
always result in correct decoding if Equation (6.51) is satisfied. 

Example 6.6 Erasure Correction 

Consider the codeword set presented in Section 6.4.3: 

OOOOfMl llOlOO 011010 ion 10 lOlOOl 011101 11(M)1I 000111 

Suppose that the codeword 1 KXIll wa.s transmitted and that the two leltmo.st digits 
were declared by the receiver to be erasures. Verify that the received flawed sequence 
xxOOl 1 can be corrected. 


Solution 


Since d 


= p + 1 - 3, the code can correct as many as p = 2 erasures. This is easily 
verified above or with Figure 6.11 by comparing the rightmost four digits of xxOOll 
with each of the allowable codewords. The codeword that was actuallv transmitted is 
closest in Hamming distance to the flawed sequence. 


6.6 USEFULNESS OF THE STANDARD ARRAY 


6.6.1 Estimating Code Capability 


The standard array can be thought of as an organizational tool, a filing cabinet that 
contains all of the possible 2 " entries in the space of ^/-tuples—nothing missing, and 
nothing replicated. At first glance, the benefits of this tool .seem limited to small 
block codes, because for code lengths beyond n = 20 , there arc millions of ^-tuples 
in the space. However, even for large codes, the standard array allows visualization 
of important performance issues, such as possible trade-offs between error correc¬ 
tion and detection, as well as bounds on error-correction capability. One such 
bound, called the Hamming hound [7], is described as 
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where ("). defined in Equation (6.16), represents the number of ways in which/ bits 
out of n may be in error. Note that the sum of the terms within the square brackets 
of Equation (6.52) yields the minimum number of rows needed in the standard 
array to correct all combinations of errors through r-bit errors. The inequality gives 
a lower bound on n - k, the number of parity bits (or the number of 2 "" ^ cosets) as 
a function of the /-bit error-correction capability of the code. Similarly, the inequal¬ 
ity can be described as giving an upper bound on the r-bit error correction capabil¬ 
ity as a function of the number of n - k parity bits (or 2" ^ cosets). For anyjji^k) 
linear block code to provide a /-bit error-correcting capability, it is a necessary 
condition that the Hamming bound be met. 

To demonstrate how the standard array provides a visualization of this 
bound, let us choose the (127, 106) BCH code as an example. The array contains ail 
2'27 ~ \ j() X n-tupics in the space.The topmost row of the array contains 
= 2"^ = S.l 1 X codewords; hence, this is the number of columns in the 
arrav. The leftmost column contains the T~^ - 2^’ = 2,097,152 coset leaders; hence, 
this is the number of rows in the array. Although the number of n-tuples and code¬ 
words is enormous, the concern is not with any individual entr>'; the primary inter- 




the 2 


est is in the number of cosets. There are 2,(J97,152 cosets, and hence there are at 
most 2,097,151 error patterns that can be corrected by this code. Next, it is shown 
how this number of cciscts dictates an upper bound on the /-bit error correcting 
capability of the code. 

Since each codeword contains 127 bits, there are 127 ways to make single er¬ 
rors. We next compute how many ways there are to make double errors, namely 
(* 2 ^) = 8,001. We move on to triple errors because thus far only a small portion of 
the total 2,097,151 correctable error-patterns have been used. There arc (^ 3 ^) = 
333,375 ways to make triple errors. On Table 6.3 these computations are listed, in¬ 
dicating that the all-zeros error pattern requires the presence of the first coset. fol¬ 
lowed by single, double, and triple errors. Also shown are the number of cosets 
required for each error type and the cumulative number of cosets necessary 
through that error type. From this table it can be seen that a (127. 106) code can 
correct all single-, double-, and triple-error patterns, which only account for 
341,504 of the available 2,097,152 cosets. The unused 1,755,648 rows are indicative 
of the fact that more error correction is possible. Indeed, it might be tempting to 
try fitting all possible 4-bit error patterns into the array. However, from Table 6.3 it 
can be seen that this is not possible, because the number of remaining cosets in the 
array is much smaller than the cumulative number of cosets required for correcting 
4-bil errors, as indicated by the last line of the table. Therefore, for this (127, 106) 
example, the code has a Hamming bound that guarantees the correction of up to 
and including all 3-bit errors. 
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TABLE 6.3 Error-Correction Bound for the (127, 106) Code 



Number of Bit Errors Number of Cosets Required Cumulative Number of Cosets Required 



0 

1 

2 

3 

4 


1 

127 

8,001 

333,37.^ 

10.334,625 


1 

128 

8.129 

341,504 

10,676,129 


6.6.2 An (n, /f) Example 

The standard array provides insight into the trade-otfs that are possible between 
error correction and detection. Consider a new (/t, k) code example, and the factors 
that dictate what values of (/?, k) should be chosen. 


1. In order to perform a nontrivial trade-off between error correction and error 

detection, it is desired that the code have an error-correcting capability of at 

least t = 2. We use Equation (6.44) for finding the minimum distance, as fol¬ 
lows: = 2/ -b 1 = 5, 

co^fe sVi-tem, it is desired that the number of data bits be at 
least A: = 2. I hus, there will be 2^ = 4 codewords. The code can now be desig¬ 
nated as an (n, 2) code. 

3. We look for the minimum value of n that will allow correcting all possible sin¬ 
gle and double errors. In this example, each of the 2" ;i-tuples in the array will 
be tabulated. The minimum value of n is desired because whenever n is incre¬ 
mented by just a single integer, the number of /i-tuples in the standard array 
doubles. It is, of course, desired that the list be of manageable size. For real 
world codes, we want the minimum n for different reasons—bandwidth effi¬ 
ciency and simplicity. If the Hamming bound is used in choosing n, then /i = 7 
could be selected. However, the dimensions of such a (7. 2) code will not 
meet our stated requirements of t = 2-bit error-correction capability and 
= 5, To see this, it is necessary to introduce another upper bound on the /-bit 
error correction capability (or This bound, called the Plorkin hound [7j. 
is described bv 



(6.53) 


In general, a linear (/i, k) code must meet all upper bounds involving error- 
correction capability (or minimum distance). For high-rate codes, if the Hamming 
bound is met, then the Plotkin bound will also be met; this was the case for the ear¬ 
lier (127, 106) code example. For low rate codes, it is the other way around [7]. 
Since this example entails a low-rate code, it is important to test error-correction 
capability via the Plotkin bound. Because = 5, it should be clear from Equation 
(6.53) that n must be 8, and therefore, the minimum dimensions of the code are 
(8, 2), in order to meet the requirements for this example. Would anyone use such 
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a low-rate double-error correcting code as this (8, 2) code? No, it would be too 
bandwidth expansive compared with more efficient codes that are available. It is 
used here for tutorial purposes, because its standard array is of manageable size. 

6.6.3 Designing the (8, 2) Code 

A natural question to ask is. How does one select codewords out of the space of 2* 
8 -tuples? There is not a single solution, but there are constraints in how choices are 
made. The following arc the elements that help point to a solution: 

1. The number of codewords is 2* = 2^ = 4. 

2. The all-zeros vector must be one of the codewords. 

3. The property of closure must apply. This property dictates that the sum of 
any two codewords in the space must yield a valid codeword in the space. 

4. Each codeword is 8 bits long. 

5. Since = 5, the weight of each codeword (except for the all-zeros code¬ 
word) must also be at least 5 (by virtue of the closure property). The weight 
of a vector is defined as the number of nonzero components in the vector. 

6 . Assume that the code is systematic, and thus the rightmost 2 bits of each 
codeword are the corresponding message bits. 

The following is a candidate assignment of codewords to messages that meets all of 
the preceeding conditions: 


Messages Codewords 


(M) 

00000000 

01 

111 HXXll 

10 

00111110 

11 

IKXlllll 


The design of the codeword set can begin in a very arbitrary way; it is only neces¬ 
sary to adhere to the properties of weight and systematic form of the code. The se- 
lection of the first few codewords is often simple. However, as the process 
continues, the selection routine becomes harder, and the choices become more 
constrained because of the need to adhere to the closure property. 

6.6.4 Error Detection versus Error Correction Trade-offs 


For the (8, 2) code system selected in the previous section, the (k x n) = (2x8) 
generator matrix can be written as 


00111110 

11110001 


Decoding starts with the computation of a syndrome, which can be thought of as 
learning the “symptom” of an error. For an (n, k) code, an (/? - k)-b\i syndrome S is 
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the product of an /i-bii received vector r, and the transpose of an {n - k) X n parity- 

check matrix H, where H is constructed so that the rows of G are orthogonal to the 

rows of H; that is GH^ = 0. For this (8, 2) example, S is a 6-bit vector, and H is a 
6x8 matrix, where 

~ I 0 U 0 Cl 0~ 

0 10 0 0 0 
0 0 1 0 0 0 
^ ^ _ 0 0 0 1 0 0 

000010 
000001 
001111 
Jill 00_ 

The syndrome for each error pattern can be calculated using Equation (6.37), 

namely 




1 ... 


where S, is one of the 2" ‘ ^ = 64 syndromes, and e, is one of the 64 coset leaders 
(error patterns) in the standard array. Figure 6.15 shows a tabulation of the stan¬ 
dard array as well as all 64 syndromes for the (8, 2) code. The set of syndromes 
were calculated by using Equation (6.37); the entries of any given row (cosel) of 
the standard array have the same syndrome. The correction of a corrupted code¬ 
word proceeds by computing its syndrome and locating the error pattern that cor¬ 
responds to that syndrome. Finally, the error pattern is modulo-2 added to the 
corrupted codeword yielding the corrected output. Equation (6.49), repeated 
below, indicates that crrt^r-detection and error-correction capabilities can be 
traded, provided that the distance relationship 


^ Qt + (3 + I 

prevails. Here, a represents the number of bit errors to be corrected. (3 represents 
the number of bit errors to be detected, and p > a. The trade-off choices available 
for the (8.2) code example arc as follows: 


Detect (p) Correct (ot) 

2 2 

3 1 

4 0 

This table shows that the (8, 2) code can be implemented to perform only eiTor cor¬ 
rection, which means that it first detects as many as p = 2 errors and then corrects 
them. If some error correction is sacrificed so that the code will only correct single 
errors, then the detection capability is increased so that all |3 = 3 errors can be de¬ 
tected. Finally, if error correction is completely sacrificed, the decoder can be im¬ 
plemented so that all |3 = 4 errors can be detected. In the ca.se of error detection 
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Syndromes 

000000 

1. 

00000000 

Standard array 

11110001 00111110 

11001111 


111100 

2. 

00000001 

11110000 

00111111 

11001110 


001111 

3. 

00000010 

11110011 

00111100 

11001101 


000001 

4. 

00000100 

11110101 

00111010 

11001011 


000010 

5. 

00001000 

11111001 

00110110 

11000111 


000100 

6. 

00010000 

11100001 

00101110 

11011111 


001000 

7. 

00100000 

11010001 

00011110 

11101111 


010000 

8. 

01000000 

10110001 

01111110 

10001111 


100000 

9. 

10000000 

01110001 

10111110 

01001111 


110011 

10. 

00000011 

11110010 

00111101 

11001100 


111101 

11. 

00000101 

11110100 

00111011 

11001010 


111110 

12. 

00001001 

11111000 

00110111 

11000110 


111000 

13. 

00010001 

11100000 

00101111 

11011110 


110100 

14. 

00100001 

11010000 

00011111 

11101110 


101100 

15. 

01000001 

10110000 

01111111 

10001110 


011100 

16. 

10000001 

01110000 

10111111 

01001110 


001110 

17. 

00000110 

11110111 

00111000 

11001001 


001101 

18. 

00001010 

11111011 

00110100 

11000101 


001011 

19. 

00010010 

11100011 

00101100 

11011101 


000111 

20. 

00100010 

11010011 

00011100 

11101101 


011111 

21. 

01000010 

10110011 

01111100 

10001101 


101111 

22. 

10000010 

01110011 

10111100 

01001101 


000011 

23. 

00001100 

11111101 

00110010 

11000011 


000101 

24. 

00010100 

11100101 

00101010 

11011011 


001001 

25. 

00100100 

11010101 

00011010 

11101011 


010001 

26. 

01000100 

10110101 

01111010 

10001011 


100001 

27. 

10000100 

01110101 

10111010 

01001011 


000110 

28. 

00011000 

11101111 

00100110 

11010111 


001010 

29. 

00101000 

11011001 

00010110 

11100111 


010010 

30. 

01001000 

10111001 

01110110 

10000111 


100010 

31. 

10001000 

01111001 

10110110 

01000111 


001100 

32. 

00110000 

11000001 

00001110 

11111111 


010100 

33. 

01010000 

10100001 

01101110 

.10011111 


100100 

34. 

10010000 

01100001 

10101110 

01011111 


011000 

35. 

01100000 

10010001 

01011110 

10101111 


101000 

36. 

10100000 

01010001 

10011110 

01101111 


110000 

37. 

11000000 

00110001 

11111110 

00001111 


110010 

38. 

00000111 

11110110 

00111001 

11001000 


110111 

39. 

00010011 

11100010 

00101101 

11011100 


111011 

40. 

00100011 

> 

11010010 

00011101 

11101100 


100011 

41. 

01000011 

10110010 

01111101 

10001100 


010011 

42. 

10000011 

01110010 

10111101 

01001100 


mill 

43. 

00001101 

11111100 

00110011 

11000010 


111001 

44. 

00010101 

11100100 

00101011 

11011010 


110101 

45. 

00100101 

11010100 

00011011 

11101010 


101101 

46. 

01000101 

10110100 

01111011 

10001010 


011101 

47. 

10000101 

01110100 

10111011 

01001010 


011110 

48. 

01000110 

10110111 

01111000 

10001001 


101110 

49. 

10000110 

01110111 

10111000 

01001001 


100101 

50. 

10010100 

01100101 

10101010 

01011011 


011001 

51. 

01100100 

10010101 

01011010 

10101011 


110001 

52. 

11000100 

00110101 

11111010 

00001011 


011010 

53. 

01101000 

10011001 

01010110 

10100111 


010110 

54. 

01011000 

10101001 

01100110 

10010111 


100110 

55. 

10011000 

01101001 

10100110 

01010111 

Figure 6.15 The syndromes 

101010 

56. 

10101000 

01011001 

10010110 

01100111 

101001 

57. 

10100100 

01010101 

10011010 

01101011 

and the standard array for the 

100111 

58. 

10100010 

01010011 

10011100 

01101101 

(8, 2) code. 
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only, the circuitry is very simple. The syndrome is computed and an error is de¬ 
tected whenever a nonzero syndrome occurs. 

For correcting single errors, the decoder can be implemented with gates [4], 
similar to the circuitry in Figure 6.12, where a received code vector r enters at two 
places. In the top part of the figure, the received digits are connected to exclusive- 
OR gates, which yield the syndrome. For any given received vector, the syndrome 
is obtained from Equation (6.35) as 



Using the values for the ( 8 , 2) code, the wiring between the received digits and 
the exclusive-OR gates in a circuit similar to the one in Figure 6.12, must be con¬ 
nected to vield 



'*2 '■4 ''s f'b r-, 



1 0 0 0 0 0 
0 1 0 0 0 0 
0 {) 1 0 0 0 
0 0 0 1 0 0 
0 0 0 0 1 0 
0 0 0 0 0 1 
0 0 1111 
111100 


Each of the Sj digits (7 = 1,... , 6 ) making up syndrome S, (/ = I,... , 64) is related 
to the input-received code vector in the following way: 


i’l 

A-4 


ri + r 


8 


'■4 + ''7 + r 


8 


^2 

■^5 


''2 '*8 

rs + r-, 


''3 + '■7 + 

'”6 + r-j 


To implement a decoder circuit similar to Figure 6.12 for the ( 8 , 2) code necessi¬ 
tates that the eight received digits be connected to six modulo -2 adders yielding the 
syndrome digits as described above. Additional modifications to the figure need to 
be made accordingly. 

If the decoder is implemented to correct only single errors; that is a = 1 and 
P = 3, then this is tantamount to drawing a line under coset 9 in Figure 6.15, and 
error correction takes place only when one of the eight syndromes associated with 
a single error appears. The decoding circuitry (similar to Figure 6.12) then trans¬ 
forms the syndrome to its corresponding error pattern. The error pattern is then 
modulo -2 added to the “potentially” corrupted received vector, yielding the cor¬ 
rected output. Additional gates are needed to test for the case in which the 
syndrome is nonzero and there is no correction designed to take place. For single¬ 
error correction, such an event happens for any of the syndromes numbered 10 
through 64. This outcome is then used to indicate an error detection. 

If the decoder is implemented to correct single and double errors, which 
means that (i = 2 errors are detected and then corrected, then this is tantamount to 
drawing a line under coset 37 in the standard array of Figure 6.15. Even though this 
( 8 , 2 ) code is capable of correcting some combination of triple errors corresponding 
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to the cosel leaders 38 through 64, a decoder is most often implemented as a 
bounded distance decoder, which means that it corrects all combinations of errors 
up to and including t errors, but no combinations of errors greater than t. The un¬ 
used error-correction capability can be applied toward some error-detection en¬ 
hancement. As before, the decoder can be implemented with gates similar to those 
shown in Figure 6.12. 

6.6.5 The Standard Array Provides Insight 


In the context of Figure 6.15, the ( 8 , 2) code satisfies the Hamming bound. That is, 
from the standard array it is recognizable that the ( 8 . 2 ) code can correct all combi-i 
nations of single and double errors. Consider the following question: Given that 
transmission takes place over a channel that always introduces errors in the form of 
a burst of 3 -bit errors and thus there is no interest in correcting single or double er¬ 
rors, wouldn’t it be possible to set up the coset leaders to correspond to only triple 
errors? It is simple to see that in a .sequence of 8 bits there are ( 3 ) = 56 ways to 
make triple errors. If we only want to correct all these 56 combinations of triple er¬ 
rors, there is sufficient room (sufficient number of cosets) in the standard array, 
since there are 64 rows. Will that work? No, it will not. For any code, the overriding 
parameter for determining error-correcting capability is d^^^- the ( 8 , 2 ) code. 


= 5 dictates that only 2 -bit error correction is possible. 

How can the standard array provide some insight as to why this scheme won’t 
work? In order for a group of A-bit error patterns to enable .r-bit error correction, 
the entire group of weighi-.v vectors must be coset leaders; that is, they must only 
occupy the leftmost column. In figure 6.15, it can be seen that all weight-1 and 
weight -2 vectors appear in the leftmost column of the standard array, and now'here 
else. Even if we forced all weight-3 vectors into row' numbers 2 through 57, we 
would find that some of these vectors would have to reappear elsewhere in the 
array (which violates a basic property of the standard array). In Figure 6.15 a 
shaded box is drawn around every one of the 56 vectors having a weight of 3. Look 
at the cosel leaders representing 3-bit error patterns, in rows 38, 41^3, 46^9, and 
52 of the standard array. Now look at the entries of the same row numbers in the 
rightmost column, where shaded boxes indicate other weight-3 vectors. Do you see 
the ambiguity that exists for each of the rows listed above, and why it is not possi¬ 
ble to correct all 3-bit error patterns with this ( 8 , 2) code? Suppose the decoder re- 
ceives the weight-3 vector I 1 0 0 1 0 0 0, located at row 38 in the rightmost column. 
This Hawed codeword could have arisen in one of tw'O ways: One w'ould be that 
codeword I 1 0 0 1 1 1 1 was sent and the 3-bit error pattern 0 0 0 0 0 1 1 1 
perturbed it; the other would be that codeword 0 0 0 0 0 0 0 0 was sent and the 
3-bii error pattern I I 0 0 1 0 0 0 perturbed it. 


6.7 CYCLIC CODES 


Binary cvclic codes are an important subclass of linear block codes. The codes are 
easily implemented with feedback shift registers; the syndrome calculation is easily 
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accomplished with similar feedback shift registers; and the underlying algebraic 
structure of a cyclic code lends itself to efficient decoding methods. An {n, k) linear 
code is called a cyclic code if it can be described by the following property. If the 
/^j-tuple LI = (W(), ih ,.... f ) is a codeword in the subspace S, then LI^‘^ = i, 

u,), »!, Uj .w„_ 2 ) obtained by an end-around shift, is also a codeword in S. 

Or, in general, LI*'* = (u„ _ . .. _ ,, /<(„ //,._ i), obtained by i 

end-around or cyclic shifts, is also a codeword in 5, 

The components of a codeword U = («,„ u^. ih ..,) can be treated as 

the coefficients of a polynomial LI(vV) as follows: 

O(A') = U(f + it2X~ -H ••• -f u„ ^X''~^ (6.54) 

. 

The polynomial function U(A') can be thought of as a “placeholder” for the digits 
of the codeword U; (hat is, an /i-tuple vector is described by a polynomial of degree 
n - i or less. The presence or absence of each term in the polynomial indicates 
the presence of a 1 or 0 in the corresponding location of the ?i-tuple. If the u„ _ j 
component is nonzero, the polynomial is of degree n - 1. The usefulness of this 
polynomial description of a codeword will become clear as we discuss the algebraic 
structure of the cyclic codes. 

6.7.1 Algebraic Structure of Cyclic Codes 

Expressing the codewords in polynomial form, the cyclic nature of the code 
manifests ilsell in the following way. If LJ(A') is an {n — l)-degree codeword 
polynomial, then the remainder resulting from dividing A^!J(A^ by A’" + 1, 

is also a codeword; that is. 

A^'U(A0 U''\A1 

or, multiplying through by A^’ + 1, 

A^XI(A1 = ^iX){r' + I) + U''>(A^) (6.55b) 

i remainder 

which can also b^described in terms of modulo arithmetic as 

U''i(A') = X'ViX) modulo (A" + 1) (6,56) 

where x modulo y is defined as the remainder obtained from dividing x by y. Let us 
demonstrate the validity of Equation (6.56) for the case of / = 1; 

U(A^) = + //,A'-h U.X^-^ ••• -K u„ 

XVJ{X) = U^X- + U.X^ + ••• 

We now add and subtract or, since we are using modulo-2 arithmetic, we add 
1 twice, as follows: 

A'U(A') = Un-\ + UqXU]X^ 4- + ••• T -f u„^iX" -f 

^ _ 

^ 

U"'(A0 
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= U<'>(A-) + + 1) 

Since U*'*(Ar) is of degree n - 1, it cannot be divided by A” + 1. Thus, from Equa¬ 
tion (6.55a), we can write 

U<'>{A') = AUCA) modulo (A" -l- 1) 

By extension, we arrive at Equation (6.56): 

U"\A) = A'U(A) modulo (A" + I) 

Example 6.7 Cyclic Shift of a Code Vector 

Let U = 1 1 0 1, for ^/ = 4. Express the codeword in polynomial form, and using 
Equation (6.56), solve for the third end-around shift of the codeword. 

Solution 

ViX) = 1 + X + X^ (polynomial is written low order to high order); 

^'U(A^) = X^ + X^ + X\ where i = 3. 

Divide A'^IJ(A^ by X"^ + 1, and solve for the remainder using polynomial division: 

X^ + 1 

X"^ + 1 )x^ + x^ + x^ 

^ _ + x^ 

x^ ^ x^ x^ 

^ 

X^ + X^ + 1 remainder U^^^( A') 

Writing the remainder low order to high order: 1 -f + A'-^, the codeword = 
10 11 is three cyclic shifts of U = 1 10 1. Remember that for binary codes, the 
addition operation is performed modulo-2, so that -t- 1 = -1, and we consequently 
do not show any minus signs in the computation. 


6.7.2 Binary Cyclic Code Properties 


We can generate a cyclic code using a generator polynomial in much the way that 
we generated a block code using a generator matrix. The generator polynomial 
g(A^ for an {n, k) cyclic code is unique and is of the form 

g(-^) = go + gi-''+ g 2 ^^ -I- ••• -b gpX’’ (6.57) 


where go niust equal 1. Every codeword polynomial in the subspace is of 

the form U(A0 = m(A0g(A^, where U(A^ is a polynomial of degree « - 1 or less. 
Therefore, the message polynomial ni(A0 is written as 


m(A') = mo + -I-m„_p_iA"" ^ ^ (6.58) 

There are 2”“^ codeword polynomials, and there are 2* code vectors in an (ai, k) 
code. Since there must be one codeword polynomial for each code vector. 
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or 


p ^ n - k 

Hence, g(A*^), ns shown in Ecjuation (6.57), must be of degree n — k, and ever^' 
codeword polynomial in the {n, k) cyclic code can be expressed as 

U(A^) = (Wf, + 4- ••• + ‘)g(*^) (6.59) 

V is said to be a valid codeword of the subspace S if, and only if g(A') divides into 
U(X) without a remainder. 

A generator polynomial g( A') of an {n, k) cyclic code is a factor of A'" + 1; that 
is, A"" + I = g(A)h(A). For example, 

A' + I = (1 + A-f A')(I + A+ X" ^ A'**) 

Using g( A) = 1 -h A + A ' as a generator polynomial of degree a? - A: = 3, we can gen¬ 
erate an (a7 . k) = (7,4) cyclic code. Or, using g(X) = 1 + A + A' -k A^ where n-k = 4 
we can generate a (7, 3) cyclic code. In summary, if g( A) is a polynomial of degree 
n - k and is a factor of A' + 1, then g(A) uniquely generates an (n. k) cyclic code. 

6.7.3 Encoding in Systematic Form 


In Section 6.4.5 we introduced the systematic form and discussed the reduction 
in complexity that makes this encoding form attractive. Let us use some of the 
algebraic properties of the cyclic code to establish a systematic encoding procedure. 
We can express the message vector in polynomial form, as follows: 

m(A) = mo = m,A-l- aaz^A" + -h m;^ (6.6(1) 

In systematic form, the message digits are utilized as part of the codeword. We can 
think of shifting the message digits into the rightmost k stages of a codeword regis¬ 
ter. and then appending the parity digits by placing them in the leftmost n - k 
stages. Therefore, we manipulate the message polynomial algebraically so that it is 

right-shifted n - k positions. If we multiply m( A) by A"■^ we get the right-shifted 
message polynomial: 

A'"‘*m(A') = + ••• + ' (6.61) 

If we ne.xt divide Equation (6.61) by g(Al. the result can be expressed as 

A'"-*m(.Y) = q(A')g(.V) + p(A') (6.62) 

where the remainder p(A) can be expressed as 

= Pi)-^ p,x+ PjX- + ••• -f 

We can also say that 

p(A) = A"~*m(A) modulo g(A) (6.63) 

Adding p(A) to both sides of Equation (6.62), using modulo-2 arithmetic, we get 

p(A) + A'-'m(A") = q(A)g(A) = U{A) (6.64) 
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The lell-hand side of Equation (6.64) is recogni/cd as a valid codeword polyno¬ 
mial. since it is a polynomial of degree n - 1 or less, and when divided by g{.¥) there 
is a zero remainder. This codeword can be expanded into its polynomial terms as 
follows: 


P(A') 


X 


ti “A 


m(X) 


+ PfX + •■■ + p„ 


k I 


A'" 


+ uhfX 


ft -k 


m^X 




-t- 


The codeword polynomial corre.sponds to the code vector 


U 


{PlhP[ . Pn-k-\- /W(|./?/i, . . . . l) 




A ) parity bits A messaiie bits 


Example 6.K Cyelic Code in Systematic Form 


^ 4 


+ m^.^X 


« - 1 


(6.65) 


Using the generator polynomial g(A') = 1 .T + A'\ generate a sysiemalic codeword 
from the (7, 4) codeword set for the message vector m = 1 0 1 1. 

Solution 


m(A) 


I -h A'2 -h A'\ n = 7, A = 4, 


n 


A 


A'’'^m(T) = A\1 t .Y‘ X^) = X- + X^ A'^’ 


Dividing X' Sii( A) by g( A”) using polynomial division, we can write 

A-’ + A-" + A^’ = (I A^-^ A"- + A") (1 + A'+ A^'^) ^ 






quotient 

q(A) 


generator 

gbV) 


1 



remainder 

P(A') 


Using Equation (6.64) yields 


U(A) = pfA) ^ A"-'m(A) = 1 + A-' -h A*'' -f A'' 




.■> 


LI = I 0 0 

parity 

bits 


1_0J_1 

message 

bits 


6.7.4 Circuit for Dividing Polynomials 


We have seen ihal the cyclic shift of a codeword polynomial and that the encoding 
of a message polynomial involves the division of one polynomial by another. Such 
an operation is readily accomplished by a dwiding circuit (feedback shift register). 
Given two polynomials V( A') and g(A"), where 


\{X) ^ r„ + v,A+ V2X" -I- ••• -h v,„A 


tn 


and 

glA') = go + giA'+ g2A'‘ + ••• + gpA^ 
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Quotient 



• • • " 1 / 
(high-order 

coefficient 

first) 


Figure 6.16 Circuit for dividing polynomials. 


such that m >p, the divider circuit of Figure 6.16 performs the polynomial division 

steps of dividing V(J 0 by g(X), thereby determining the quotient and remainder 
terms: 


vw 

gW 


q{X) + 


gW 


The stages of the register are first initialized by being filled with zeros. The first 
p shifts enter the most significant (higher-order) coefficients of V(A0. After the pih 
shift, the quotient output is g~^ this is the highest-order term in the quotient. For 
each quotient coefficient the polynomial q,g(X) must be subtracted from the div¬ 
idend. The feedback connections in Figure 6.16 perform this subtraction. The dif¬ 
ference between the leftmost p terms remaining in the dividend and the feedback 
terms qig{X) is formed on each shift of the circuit and appears as the contents of 
the register. At each shift of the register, the difference is shifted one stage; the 
highest-order term (which by construction is zero) is shifted out, while the next sig¬ 
nificant coefficient of \{X) is shifted in. After m + 1 total shifts into the register, 

the quotient has been serially presented at the output and the remainder resides in 
the register. 

Example 6.9 Dividing Circuit 

Use a dividing circuit of the form shown in Figure 6.16 to divide V(A^ = X^ + X^ + X^ 

(V = 0001 01 l)by g(.^ = (1 -I- A" + A'^). Find the quotient and remainder terms. 

Compare the circuit implementation to the polynomial division steps performed by 
hand. 


Solution 

The dividing circuit needs to perform the following operation: 


A^^ + A^^ + X^ 
\ + X+ X"^ 


q(^) + 


P(^) 

1 + A^-f A^^ 
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Feedback polynomial 



0 0 0 1 0 1 1 


Output 


Figure 6.17 Dividing circuit for Example 6.9. 


The required feedback shift register, following the general form of Figure 6.16, is 
shown in Figure 6.17. Assume that the register contents are initially zero. The opera¬ 
tional steps of the circuit are as follows: 


Input queue 


0 0 0 1 01 1 
0001 01 
000 1 0 
000 1 
000 
0 0 
0 


Shift number 


Register contents 


Outout and Feedback 


0 000 

1 100 0 

2 110 0 

3 011 0 

4 oil 1 

5 111 1 

6 101 1 

7 10 0 1 


After the fourth shift, the quotient coefficients serially presented at the output are 
seen to be 1 1 1 1, or the quotient polynomial is q(A^ = \ + X + + X^. The remainder 

coefficients {pj are 1 0 0, or the remainder polynomial p(X) = 1. In summary, the 

circuit computation V(30/g(J0 is seen to be 


X^ + X^ + X^ 

1 -I- x+ X^ 


1 + x+ x^ + x^ + 


1 

1 + 


The polynomial division steps are as follows: 




X^ + X+ I 


Output after shift number: 
4 5 6 7 


i i i i 

X^ X" + X + 1 

)x^ + x^ 

x^ + x^ ^ x^ ^ - 

A''*' + X^^ - 

A'^ + A'-'* + X^ - 

X^ + X'^ X ^ 

A'^ ^ X^ 

X'^ + A' 1 


feedback after 4th shift 
_ register after 4th shift 

feedback after 5th shift 

- register after 5th shift 
feedback after 6th shift 

- register after 6th shift 
feedback after 7th shift 


1 -register after 7th shift 

(remainder) 
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6.7.5 Systematic Encoding with an 

{n - ^-Stage Shift Register 

The encoding of a cyclic code in systematic form has been shown, in Section 6.7.3, 
to involve the computation of parity bits as the result of the formation of 
X”~^m{X) modulo %{X), in other words, the division of an upshifted (right shifted) 
message polynomial by a generator polynomial g(JO- I’he need for upshifting is to 
make room for the parity bits, which are appended to the message bits, yielding the 
code vector in systematic form. Upshifting the message bits by /] - positions is a 
trivial operation and is not really performed as part of the dividing circuit. Instead, 
only the parity bits are computed; they are then placed in the appropriate location 
alongside the message bits. The parity polynomial is the remainder after dividing by 
the generator polynomial; it is available in the register after n shifts through the 
(n - A)-stage feedback register shown in Figure 6.17. Notice that the first n - k 
shifts through the register are simply filling the register. We cannot have any feed¬ 
back until the rightmost stage has been filled; we therefore can shorten the shifting 
cycle by loading the input data to the output of the last stage, as shown in Figure 
6.18. Further, the feedback term into the leftmost stage is the sum of the input and 
the rightmost stage. We guarantee that this sum is generated by ensuring that 
Si) = Sn-k = 1 for any generator polynomial g(A0. The circuit feedback connections 
correspond to the coefficients of the generator polynomial, which is written as 

g(A') = 1 + ^1^^+ g2X^ + ■■■ + (6.66) 

The following steps describe the encoding procedure used with the Figure 6.18 
encoder: 



V — 

n-k shift register stages 





m(X) o 


Output 


V{X) 


Switch 2 


Figure 6.18 Encoding with an (n- /c)-stage shift register. 
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1. Switch 1 is closed during the first k shifts, to allow transmission of the 
message bits into ihcn- k stage encoding shift register. 

2. Switch 2 is in the down position to allow transmission of the message bits 
directly to an output register during the first k shifts. 

3. After transmission of the A:th message bit, switch 1 is opened and switch 2 is 
moved to the up position. 

4. The remaining n - k shifts clear the encoding register by moving the parity 
bits to the output register. 

5. The total number of shifts is equal to n, and the contents of the output regis¬ 
ter is the codeword polynomial p(20 + 

Example 6.10 Systematic Encoding of a Cyclic Code 

Use a feedback shift register of the form shown in Figure 6.18 to encode the message 
vector m = 1 0 1 1 into a (7, 4) codeword using the generator polynomial g(20 = 

l+X+X\ 

Solution 

m = 1 0 1 1 

m(A') = \+ X^ ^ X^ 

A^"-'m(2r) = X-m{X) = X^ + X^ + X^ 

X'^-^miX) = q{XMX) + p{X) 

p(A') = [X^ + + X^) modulo (1 + + X^) 

For the {n- k) = 3-stage encoding shift register shown in Figure 6.19, the operational 
steps are as follows: 


Assume: g(X)=^+X + X^ 



Output 


Figure 6.19 Example of encoding a (7, 4) cyclic code with an 
(n - /c)-stage shift register. 
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Input queue 


Shift number 


Regisier contents 


Output 



1 0 I ] 0 

101 1 

10 2 

1 3 


4 


000 


110 1 

10 1 I 

1 00 0 

10 0 1 


After the fourth shill, switch I is opened, switch 2 is moved to the up position, and the 
parity bits contained in the register are shifted to the output. The output codeword is 
U = 1 0 0 1 0 1 I, or in polynomial form, U(A0 = 1 + A"- + X"' + 


6.7.6 Error Detection with an 

{n - /()-Stage Shift Register 


A transmitted codeword may be perturbed by noise, and hence the vector received 
may be a corrupted version of the transmitted codeword. Let us assume that a 
codeword with polynomial representation LI(J0 transmitted and that a vector 
with polynomial representation Z(X) is received. Since iJ{X) is a code polynomial, 
it must be a multiple of the generator polynomial g(A^): that is. 

U(A') = iii(A')g(A’) (6.67) 

and Z(X). the corrupted version of U(A0, can be written as 

Z{X) = V(X) Te(A') (6.6S) 

where e(X) is the error pattern polynomial. The decoder tests whether Z(A^) is a 

codeword polynomial, that is, whether it is divisible by g(A'), with a zero remainder. 

This is accomplished by calculating the syndrome of the received polynomial. The 

syndrome S(Ar) is equal to the remainder resulting from dividing Z(A') by g(A0; 
that is, 


Z(X) = q{X)^{X) + S(A'') (6.69) 

where S(A0 is a polynomial of degree n — k - \ or less. Thus, the syndrome is an 
(n - /:)-tuple. By combing Equations (6.67) to (6.69), we obtain 


t(X) 


[m{X) + q(X)]^,(X) tS(A') 


(6.70) 


By comparing Equations (6.69) and (6.70), we see that the syndrome S(A"), ob- 

r of Z(X) modulo g(A"), is exactly the same polynomial ob- 




information 


The 


cal to the encoding circuit used at the transmitter. An example of syndrome 
calculation with an n — k shift register is shown in Figure 6.20 using the code vector 
generated in Example 6.10. Switch 1 is initially closed, and switch 2 is open. The re¬ 
ceived vector is shifted into the register input, with all stages initially set to zero. 
After the entire received vector has been entered into the shift register, the con- 
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Syndrome 

output 


Figure 6.20 Example of syndrome calculation with an {n - /f)-stage shift register. 


tents of the register is the syndrome. Switch 1 is then opened and switch 2 is closed, 
so that the syndrome vector can be shifted out of the register. The operational 
steps of the decoder are as follows: 


Input queue Shift number Register contents 



10 0 10 1 1 0 0 0 0 

100101 1 100 

1 0 010 2 no 

1001 3 011 

100 4 011 

10 5 111 

1 6 101 



0 0 0 Syndrome 


If the syndrome is an all-zeros vector, the received vector is assumed to be a valid 
codeword. If the syndrome is a nonzero vector, the received vector is a perturbed 
codeword and errors have been detected; such errors can be corrected by adding 
the error vector (indicated by the syndrome) to the received vector, similar to the 
procedure described in Section 6.4.8. This method of decoding is useful for simple 
codes. More complex codes require the use of algebraic techniques to obtain prac¬ 
tical decoders [6, 8]. 


6.8 WELL-KNOWN BLOCK CODES 

6.8.1 Hamming Codes 

Hamming codes are a simple class of block codes characterized by the structure 

{n,k) = (2"* - 1,2"’ - 1 - m) (6.71) 

where m = 2, 3,_These codes have a minimum distance of 3 and thus, from 

Equations (6.44) and (6.47), they are capable of correcting all single errors or de¬ 
tecting all combinations of two or fewer errors within a block. Syndrome decoding 
is especially suited for Hamming codes. In fact, the syndrome can be formed to act 
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as a binary pointer to identify the error location [5]. Although Hamming codes are 

not very powerful, they belong to a very limited class of block codes known as 
perfect codes, described in Section 6.5.4. 

Assuming hard decision decoding, the bit error probability can be written, 
from Equation (6.46), as 

^ 2 j("j) Ml - P)"'' (6.72) 

where p is the channel symbol error probability (transition probability on the 

binary symmetric channel). In place of Equation (6.72) we can use the following 

equivalent equation. Its identity with Equation (6.72) is proven in Appendix D, 
Equation (D.16): 

Pb ^ P -/?(! -/?)"■' (6.73) 

Figure 6.21 is a plot of the decoded versus channel-symbol error probability, 
illustrating the comparative performance for different types of block codes. For the 



Channel symbol error probability, p 


Figure 6.21 Bit error probability versus channel symbol error probability 
for several block codes. 
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Hamming codes, the plots are shown for m = 3, 4, and 5, or (n, k) = (7, 4), (15, 11), 
and (31, 26). For performance over a Gaussian channel using coherently demodu¬ 
lated BPSK, we can express the channel symbol error probability in terms of EJNq, 
similar to Equation (4.79), as 



where E^JNq is the code symbol energy per noise spectral density, and where Q{x) 
is as defined in Equation (3.43). To relate E^/Nq to information bit energy per noise 
spectral density (Ef^/No), we use 



For Hamming codes, Equation (6.75) becomes 



Combining Equation (6.73), (6.74), and (6.76), Py can be expressed as a function of 
EiJNq for coherently demodulated BPSK over a Gaussian channel. The results are 
plotted in Figure 6.22 for different types of block codes. For the Hamming codes, 
plots are shown for {n, k) = (7,4), (15,11), and (31, 26). 

Example 6.11 Error Probability for Modulated and Coded Signals 

A coded orthogonal BFSK modulated signal is transmitted over a Gaussian channel. 
The signal is noncoherently detected and hard-decision decoded. Find the decoded bit 
error probability if the coding is a Hamming (7, 4) block code and the received E^^INq 
is equal to 20. 

Solution 

First we need to find EJNq using Equation (6.75): 



= 11.43 


Then, for coded noncoherent BFSK, we can relate the probability of a channel symbol 
error to E^INq, similar to Equation (4.96), as follows 


P 


1 

2 
1 
2 


exp 


Ec 


exp 


2A^o 

11.43 

2 


1.6 X 10 


-3 


Using this result in Equation (6.73), we solve for the probability of a decoded bit 
error, as follows: 


P 


B 


P - P(1 - pf ^ 1-6 X 10 


-5 
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(dB) 


Figure 6.22 Pg versus EJNq for coherently demodulated BPSK over a 
Gaussian channel for several block codes. 


6,8.2 Extended Goiay Code 


One of the more useful block codes is the binary (24, 12) extended Goiay code, 
which is formed by adding an overall parity bit to the perfect (23, 12) code, known 
as the Goiay code. This added parity bit increases the minimum distance from 
7 to 8 and produces alrate | code, which is easier to implement (with regard to sys¬ 
tem clocks) than the rate 12/23 original Goiay code. Extended Goiay codes are 


considerably more powerful than the 


Hamming codes described in the preceding 


section. The price paid for the improved performance is a more complex decoder, a 


lower code rate, and hence a larger bandwidth expansion. 
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Since f/^in = ^ for the extended Golay code, we see from Equation (6.44) that 
the code is guaranteed to correct all triple errors. The decoder can additionally be 
designed to correct some hut not all four-error patterns. Since only 16.7% of the 
four-error patterns can be corrected, the decoder, for the sake of simplicity, is usu¬ 
ally designed to only correct three-error patterns [5]. Assuming hard decision de¬ 
coding, the bit error probability for the extended Golay code can be written as a 
function of the channel symbol error probability/^ from Equation (6.46), as follows: 

1 /lA \ 

^ 2 i{j) /''■(' - P)-"'' ( 6 . 77 ) 

The plot of Equation (6.77) is shown in Figure 6.21; the error performance of the 
extended Golay code is seen to be significantly better than that of the Hamming 
codes. Combining Equations (6.77), (6.74), and (6.75), we can relate Pfj versus 
Ei,/Nq for coherently demodulated BPSK with extended Golay coding over a 
Gaussian channel. The result is plotted in Figure 6.22. 

6.8.3 BCH Codes 

Bose-Chadhuri-Hocqucnghem (BCH) codes are a generalization of Hamming 
codes that allow multiple error correction. They are a powerful class of cyclic codes 
that provide a large selection of block lengths, code rates, alphabet sizes, and error- 
correcting capability. Table 6.4 lists some code generators g(.r) commonly used for 
the construction of BCH codes [ 8 ] for various values of n, k, and t, up to a block 
length of 255. The coefficients of g(.v) are presented as octal numbers arranged so 
that when they are converted to binary digits the rightmost digit corresponds to the 
zero-degree coefficient of g(.v). From Table 6.4, one can easily verify' a cyclic code 
property—the generator polynomial is of degree n - k. BCH codes are important 
because at block lengths of a few hundred, the BCH codes outperform all other 
block codes with the same block length and code rate. The most commonly used 
BCH codes employ a binary alphabet and a codeword block length of n = 2'*' - 1. 
where m = 3, 4. 

The title of Table 6.4 indicates that the generators shown are for those BCH 
codes known as primitive codes. The term “primitive’" is a number-theoretic con¬ 
cept requiring an algebraic development [7, 10-11], which is presented in Section 
8.1.4. In Figures 6.21 and 6.22 are plotted error performance curves of two BCH 
codes (127, 64) and (127, 36), to illustrate comparative performance. Assuming 
hard decision decoding, the Pf^ versus channel error probability is shown in Figure 
6.21. The Pn versus EV/V(, for coherently demodulated BPSK over a Gaussian 
channel is shown in Figure 6.22. The curves in Figure 6.22 seem to depart from our 
expectations. They each have the same block size, yet the more redundant (127,36) 
code does not exhibit as much coding gain as does the less redundant (127, 64) 
code. It has been shown that a relatively broad maximum of coding gain versus 
code rate for fixed n occurs roughly between coding rates of 5 and 3 for BCH codes 
112). Performance over a Gaussian channel degrades substantially at very high or 
very low rales [ 11 ]. 
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Figure 6.23 represents computed performance of BCH codes (13] using coher¬ 
ently demodulated BPSK with both hard- and soft-decision decoding. Soft-decision 
decoding is not usually used with block codes because of its complexity. However, 
whenever it is implemented, it offers an approximate 2-dB coding gain over hard- 
decision decoding. For a given code rate, the decoded error probability is known 
to improve with increasing block length n [4], Thus, for a given code rate, it is 



4 5 6 7 8 9 10 

Eh/No (dB) 


Figure 6.23 Pb versus Et,/No for coherently demodulated BPSK 
over a Gaussian channel using BCH codes. {Reprinted with permis¬ 
sion from L. J. Weng, “Soft and Hard Decoding Performance Com¬ 
parisons for BCH Codes," Proc. Ini Conf. Common., 1979, Fig. 3, 
p. 25.5.5. © 1979 IEEE.) 
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interesting to consider the block length that would be required for the hard- 
decision-decoding performance to be comparable to the sofl-decision-decoding per¬ 
formance. In Figure 6.23, the BCH codes shown all have ctxie rales of approximately k. 
From the figure [13J it appears that for a fixed code rate, the hard-decision-dccoded 
BCH code of length 8 times n or longer has a belter performance (for Pp of about 10“^ 
or Ic.ss) than that of a soft-decision-decoded BCH code of length n. One special subcla.ss 
of the BCH codes (the discovery of which preceded the BCH codes) is the particularly 
useful nonh ’mary set called Reed-Solonum codes. They are described in Section 8.1. 


6.9 CONCLUSION 

In this chapter we have explored the general goals of channel coding, leading to im¬ 
proved performance (error probability, ot capacity) at a cost in bandwidth. 

We partitioned channel coding into two study groups: waveform coding and struc¬ 
tured sequences. Waveform coding represents a transformation of waveforms into 
improved waveforms, such that the distance properties are improved over those of 
the original waveforms. Structured sequences involve the addition of redundant 
digits to the data, such that the redundant digits can then be employed for delect¬ 
ing and/or correcting specific error patterns. 

We also closely examined linear block codes. Geometric analogies can be 
drawn between the coding and modulation disciplines. They both seek to pack the 
signal space efficiently and to maximize the distance between signals in the signal¬ 
ing set. Within block codes, we looked at cyclic codes, which are relatively easy to 
implement using modem integrated circuit techniques. We considered the polyno¬ 
mial representation of codes and the correspondence between the polynomial 
structure, the necessary algebraic operations, and the hardware implementation. 
Finally, we looked at performance details of some of the well-known block codes. 
Other coding subjects are treated in later chapters. In Chapter 7 we study the large 
class of convolutional codes; in Chapter 8 we discuss Reed-Solomon codes, 
concatenated codes, and turbo codes; and in Chapter 9 we examine trellis-coded 
modulation. 
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PROBLEMS 


6.1. Design an (n, k) single-parity code that will detect all 1-, 3-, 5-, and 7-error patterns 
in a block. Show the values of n and k, and find the probability of an undetected 
block error if the probability of channel symbol error is 10"^. 

6.2. Calculate the probability of message error for a 12-bit data sequence encoded with a 
(24,12) linear block code. Assume that the code corrects all 1-bit and 2-bit error pat¬ 
terns and assume that it corrects no error patterns with more than two errors. Also, 
assume that the probability of a channel symbol error is 10"^. 

6.3. Consider a (127, 92) linear block code capable of triple error corrections. 

(a) What is the probability of message error for an uncoded block of 92 bits if the 
channel symbol error probability is 10"‘^? 

(b) What is the probability of message error when using the (127, 92) block code if 
the channel symbol error probability of 10"^? 

6.4. Calculate the improvement in probability of message error relative to an uncoded 
transmission for a (24, 12) double-error-correcting linear block code. Assume that 
coherent BPSK modulation is used and that the received EfJNQ= 10 dB. 

6.5. Consider a (24, 12) linear block code capable of double-error corrections. Assume 
that a noncoherently detected binary orthogonal frequency-shift keying (BFSK) 
modulation format is used and that the received EfJNQ = 14 dB. 

(a) Does the code provide any improvement in probability of message error? If it 
does, how much? If it does not, explain why not. 

(b) Repeat part (a) with = 10 dB. 

6.6. The telephone company uses a “best-of-five” encoder for some of its digital data 
channels. In this system every data bit is repeated five times, and at the receiver, a 
majority vote decides the value of each data bit. If the uncoded probability of bit 
error is 10~^, calculate the decoded bit-error probability when using such a best-of- 
five code. 
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6.7. The minimum distance for a particular linear block code is 11. Find the maximum 
error-correcting capability, the maximum error-detecting capability^ and the maxi¬ 
mum erasure-correcting capability in a block length. 

6.8. Consider a (7,4) code whose generator matrix is 

11110 0 0 
10 10 10 0 
0 110 0 10 
1 10 0 0 0 1 

(a) Find all the codewords of the code. 

(b) FindH, the parity-check matrix of the code. 

(c) Compute the syndrome for the received vector 1 1 0 1 1 0 1. Is this a valid code 
vector? 

(d) What is the error-correcting capability of the code? 

(e) What is the error-detecting capability of the code? 

6.9. Consider a systematic block code whose parity-check equations are 

Pi = ffU 

Pi = ffh + ttti 

Pi = + W2 ntj 

Pa - W4 

where arc message digits and p, are check digits. 

(a) Find the generator matrix and the parity-check matrix for this code. 

(b) How many errors can the code correct? 

(c) Is the vector 10101010 a codeword? 

(d) Is the vector 01011100 a codeword? 

6.10. Consider the linear block code with the codeword defined bv 

U = mi + m2 ^ m^ + m^, mi -t- m^ + ujs* + rriy-^ mi, + m^, 
mi + mi + mj, mi, m3, m4, 

(a) Show the generator matrix. 

(b) Show the parity-check matrix, 

(c) Find n, k, and 

6.11. Design an {n, k) = (5,2) linear block code. 

(a) Choose the codewords to be in systematic form, and choose them with the goal 
of maximizing 

(b) Find the generator matrix for the codeword set. 

(c) Calculate the parity-check matrix. 

(d) Enter all of the n-luples into a standard array. 

(e) What are the error-correcting and error-detecting capabilities of the code? 

(f) Make a syndrome table for the correctable error patterns. 

6.12. Consider the (5, 1) repetition code, which consists of the two codewords (XKHlO and 

11111, corresponding to message 0 and 1, respectively. Derive the standard array for 
this code. Is this a perfect code? 

6.13. Design a (3, 1) code that will correct all single-error patterns. Choose the codeword 

.set and show the standard array. 
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6.14. Is a (7, 3) code a perfect code? Is a (7, 4) code a perfect code? Is a (15, 11) code a 

perfect code? Justify your answers. 


6.15. A (15, 11) linear block code can be defined by the following parity array: 



0 I I 
1 0 I 
0 U I 
1 I 0 

0 I 0 
1 0 0 
1 1 1 
1 1 0 
1 0 1 
0 1 1 
1 1 1 


(a) Show the paritv-check matrix for this code. 

(b) List the coset leaders from the standard array. Is this eode a perfect code? Justify 
your answer, 

(c) A received vector is V = 0 I I 1 1 I 0 0 1 0 1 10 1 1. Compute the syndrome. 
Assuming that a single bit error has been made, find the correct codeword. 

(d) How many erasures can this code correct? Explain. 

6.16. Is it possible that a nonzero error pattern can produce a syndrome of S = 0? If yes, 

how many such error patterns can give this result for an {n, k) code? Use Figure 6.11 


to justify your i 




6.17. Determine which, if any, of the following polynomials can generate a cyclic code 

with codeword length n < 1. Find the (n. k) values of any such codes that can be 
generated. 

(a) 1 + 

(b) 1 + A^- + A'^ 

(c) 1 + 

(d) 1 ^X-^-X^^X^ 

(e) I-hX^-^A ' 


6.18. Encode the message 1 0 1 in systematic form using polynomial division and the 

generator %iX) = 1 + X^ X" + A^". 

6.19. Design a feedback shift register encoder for an (8, 5) cyclic code with a generator 

g(.r) = 1 -f A'+ A^^ X^. Use the encoder to find the codeword for the message 10 10 1 
in systematic form. 


6.20. In Figure P6.I the signal is differentially coherent FSK (DPSK), the encoded symbol 

rate is 10,000 code symbols per second, and the decoder is a single-error-correcting 
(7, 4) decoder. Is a predetection signal-to-noise spectral density ratio of = 48 
dBW sufficient to provide a probability of message error of 10at the output? 


Figure P6.1 


Input 



Output 
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Justify your answer. Assume that a message block contains 4 data bits and that any 
single-error pattern in a block length of 7 bits can be corrected. 

6.21. A (15, 5) cyclic code has a generator polynomial as follows: 

g(A') ^ 1 + X" + A'* + .V”' 


(a) Draw a diagram of an encoder for this code. 

(b) Find the code polynomial (in systematic form) for the message m(A^) - 

1 + a:- -K A'-*. 

(c) Is VIA") =1 + A'^ + A"’ + A'^ A*** a code polynomial in this system? Justify your 


answer. 


6.22. Consider the (15.11) cyclic code generated by g( A") = I + A' -i- A 

(a) Devise a feedback register encoder and decoder for this code. 

(b) Illustrate the encoding procedure with the message vector 11()01101011 by listing 
the states of the register (the rightmost bit is the earliest bit). 

(c) Repeal part (b) for the decoding procedure. 

6.23. For a fixed probability of channel symbol error, the probability of bit error for a 

Hamming (15.11) code is worse than that for a Hamming (7, 4) code. Explain why. 
What, then, is the advantage of the (15.11) code? What basic trade-off is involved? 

6.24. A (6.3. 36) BCH code can correct five errors. Nine blocks of a (7, 4) code can correct 


nine errors. Both codes have the same code rate. 

(a) The (7, 4) code can correct more errors. Is it more powerful? Explain. 

(b) Compare the two codes when live errors occur randomly in 63 bits. 

6.25. Information from a .source is organized in .36-bit messages that are to be transmitted 

over an AWGN channel using noncoherently delected BFSK modulation, 

(a) If no error control coding is used, compute the EiJNq required to provide a 
message error probability of KJ ^ 

(b) Consider the use of a (127, 36) linear block code (minimum distance is 31) in the 

transmission of these messages. Compute the coding gain for this code for a mes¬ 
sage error probability of 10 ^ {Hint: The coding gain is defined as the difference 
between the required without coding and the E/,/vVo required with coding.) 

6.26. (a) Consider a data sequence encoded with a (127, 64) BCH code and then 

modulated using coherent 16-ary PSK. If the received i^ If^ Ibe 

MPSK probability of symbol error, the probability of code-bit error (assuming 
that a Gray code is used for symbol-to-bit assignment), and the probability of 
information-bit error. 

(b) For the same probability of information-bit error found in part (a), determine 
the value of required if the modulation in pari (a) is changed to coherent 



orthogonal 16-ary FSK. Explain the difference. 

A message consists of English text (assume that each word in the message contains 
six letters). Each letter is encoded using the 7-bit ASCII character code. Tlius, each 


word of text consists of a 42-bit sequence. The message is to be transmitted over a 
channel having a .symbol error probability of 10"\ 

(a) What is the probability that a word will be received in error? 

(b) If a repetition code is used such that each letter in each word is repeated three 
limes, and at the receiver, majority voting is used to decode the message, what is 
the probability that a decoded word will be in error? 

(c) If a (126, 42) BCH code with error-correcting capability of r = 14 is used to 
encode each 42-bit word, what is the probability that a decoded word will be in 


error? 
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(d) For a real system, it is not fair to compare uncoded versus coded message error 
performance on the basis of a fixed probability of channel symbol error, since 
this implies a fixed level of received for all choices of coding (or lack of 

coding). Therefore, repeat parts (a), (b), and (c) under the condition that Ihc 
channel symbol error probability is determined by a received of 12 dB. 

where is the information bit energy per noise spectral density. .Assume 

that the information rate must be the same for all choices of coding or lack of 
coding. Also assume that noncoherent orthogonal binary FSK modulation is 


coding. Also assume that noncoherent orthogonal binary FSK modulation is 
used over an AWGN channel. 

(e) Discuss the relative error performance capabilities of the above coding schemes 


under the two postulated conditions—fixed channel symbol error probability, 
and fixed Under what circumstances can a repetition code offer error 

performance improvement? When will it cause performance degradation? 

6.28. A 5-bit data sequence is transformed into an orthogonal coded sequence using a 

Hadamard matrix. Coherent detection is performed over a codeword interval of 
time, as shown in Figure 6.5, Compute the coding gain relative to transmitting the 
data one bit at a time using BPSK. 

6.29. For the (S, 2) code described in Section 6.6.3, verify that the values given for the 

generator matrix, the parity-check matrix, and the syndrome vectors for each of the 
cosets from 1 through 10. are valid. 

6.30. Using exclusivc-OR gates and AND gates, implement a decoder circuit, similar to 

the one shown in Figure 6.12, that will perform error correction for all single-error 
patterns of the (8, 2) code described by the coset leaders 2 through 9 in Figure 6.15. 

6.31. Explain in detail how' you could use exclusivc-OR gates and AND gales to imple¬ 
ment a decoder circuit, similar to the one showm in Figure 6.12, that performs error 
correction for all single and double-error patterns of the (8, 2) code, and performs 
error detection for the triple-error patterns (coset leaders or rows 38 through 64). 

6.32. Verify that all of the BCH codes of length /; = 31, shown in Table 6.4, meet the 

Hamming bound and the Plotkin bound. 


6.33. When encoding an all-zeros message block, the result is an all-zeros codeword. It is 

generally undesirable to transmit long runs of such zeros. One cyclic encoding 
echnique that avoids such tran.smissions involves preloading the shift-register 
stages with ones instead of zeros, prior to encoding. The resulting “pseudoparily” is 
then guaranteed to contain some ones. At the decoder, there needs to be a reversal 
of the pseudoparity before the decoding operation starts. Devise a general scheme 
for reversing the pseudo-parity at any such cyclic decoder. Use a (7, 4) BCH 
encoder, preloaded with ones, to encode the message 10 11 (right-most bit is 
earliest). Then demonstrate that your reversal scheme, which is applied prior to 
decoding, yields the correct decoded message. 


6.34. (a) Using the generator polynomial for the (15. 5) cyclic code in Problem 6.21, 

encode the message sequence 110 1 1 in systematic form. Show the resulting 
codeword polynomial. What property characterizes the degree of the generator 
polynomial? 

(b) Consider that the received codeword is corrupted by an error pattern e{X) = 

-I- Show the corrupted codeword polynomial. 

(c) Form the syndrome polynomial by using the generator and received-codeword 
polynomials. 

Form the syndrome polynomial by using the generator and error-pattern polyno¬ 
mials, and verify that this is the same syndrome computed in part (c). 
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(e) Explain why ihe syndrome computations in parts (c) and (d) must yield identical 
results. 

(0 Using the properties u( the standard array of a (15,5) linear block code, find the 
maximum amount of error correction possible for a code with these parameters. 
Is a (15,5) code a perfect code? 

(g) If we want to implement the (15. 5) cyclic code to simultaneously correct two 
erasures and still perform error correction, how much error correction would 
have to be sacrificed? 


QUESTIONS 


6.1. Describe four types of trade-offs that can be accomplished by using an error-correct¬ 
ing code. (See Section 6.3.4.) 

6.2. In a real-time communication system, achieving coding gain by adding redundancy 
costs bandwidth. What is the usual cost for achieving coding gain in a non-real-time 
communication system? (See Section 6.3.4.2.) 


6.3. In a real-time communication system, added redundancy means faster signaling, less 
energy per channel symbol, and more errors out of the demodulator. In the face of 
such degradation, explain how coding gain is achieved. (See Example 6.2.) 

6.4. Why do error-correcting codes typically yield error-performance degradation at low 
values of E^JN^p. (See Section 6,3,4.6.) 


6ii, Describe the process of syndrome testing, error detection and correction in the 
context of a medical analogy. (See Section 6.4.8.4.) 

6.6. Of what use is the standard array in understanding a block code, and in evaluating its 
capability? (See Section 6.6.5.) 


EXERCISES 

Using the Companion CD, run the exercises associated with Chapter 6. 
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